argocd, my favorite gitops solution, allows to send notification about event (such as sync failed) on several targets: slack email HTTP aka webhook On the other side, elasticsearch/kibana, my favorite data visualization solution. Let’s connect them together! Argo notification configuration As usally with argo components, configuration is make via a config-map (you can…

When monitoring kubernetes workloads, most of the time you go for prometheus / grafana, because: small footprint, a few GB and CPUs integration with CRD ServiceMonitor, PrometheusRule kube-prometheus-stack Helm chart The concept of https://github.com/prometheus-operator/prometheus-operator is simple and powerful, a Prometheus and Alertmanager instance configured by kubernetes CR ServiceMonitor, PrometheusRule.

At X.com, we love elasticsearch, we use it to power our search engines, and a lot for analytics stuff. On other side, our genius data analyst, has bring Apache Spark in the company, we used to use a home made ETL, called Djobi to export our HDFS data to elasticsearch…

At x.com analytics, we are moving our data lake from Hadoop to kubernetes + S3 Ceph storage. On Hadoop, we used to run Djobi, a simple java application to run Apache Spark data pipeline jobs. Djobi was really cool it did: workflow architecture (run X jobs from same YAML templates…

Snappy is a very good compression algo., used by Spark to write & read smaller parquet files. Spark use a java library “xerial” JNI-based implementation to achieve comparable performance to the native C++ version. Snappy-java contains native libraries built for Window/Mac/Linux, etc. snappy-java loads one of these libraries according to…

You are using elastic APM (https://www.elastic.co/fr/observability/application-performance-monitoring) to watch whats happen inside your application. Basically, APM will write into elasticsearch 2 kind of documents: transaction span In a case of a web application, transaction will represent the end-user HTTP request, and span any interaction your application does with DB, cache, micro-service…

Dans cette courte story, je vais vous expliqer comment bien versionner des indices elasticsearch, en utilisant les alias. Pourquoi faire ? On peut presque traiter la data comme du code, étant donné que la data généralement est issue au final de code de programmation ! Versionner du contenu c’est : garder le contenu …

I am a big fan of Ansible, as a good / lazy engineer, I love making everything automatic! Argo CD is a declarative, GitOps continuous delivery tool for Kubernetes. Basically, you declare applications such as Helm release in a git repository, then argoCD will transform this as K8S object, pretty…

Petit retour d’expérience qui a mal tourné….. Je dédie ce post medium à mon collègue et ami, Clément Fourrel, Lead Senior Developper chez subitoLabs, que j’ai toujours le plaisir de revoir à St Barth. Un beau matin d’hiver, je regarde mon code YAML gitops, relié à argoCD pour déploiement automatique…