وصف الوظيفة
We are seeking a skilled Senior Data Engineer with deep expertise in ClickHouse and streaming data, and a passion for building scalable real-time analytics solutions. In this role, you will design, develop, and optimize our data pipelines and analytics infrastructure, empowering our teams to harness real-time insights that enhance customer experience and drive business growth.
Key Responsibilities
● Design, implement, maintain and document highly scalable data pipelines for real-time and batch processing.
● Build and optimize data systems to support accurate, low-latency analytics and reporting use cases.
● Develop and maintain solutions for streaming and serverless data processing.
● Collaborate with cross-functional teams to implement and support end-to-end analytics workflows.
● Ensure data quality, reliability, and performance across the platform.
● Monitor, troubleshoot, and optimize data infrastructure to maintain high availability.
● Mentor junior engineers and contribute to the continuous improvement of engineering practices.
متطلبات الوظيفة
● 5+ years of experience in data engineering or related fields.
● Strong expertise in ClickHouse: experience in designing schemas and jobs, optimizing data ingestion as well as queries, managing clusters, etc.
● Proven experience in real-time data processing and enrichment using tools like Apache Kafka, Apache Flink, Apache Spark Streaming, Serverless technologies like AWS Lambda, GCP Functions, etc.
● Deep understanding of Distributed systems architecture and design, with a focus on scalability and resilience, especially in relation to data processing
● Proficiency in programming languages like Python, or Java
● Hands-on experience with cloud platforms (e.g., AWS, GCP, or Azure).
● Familiarity with containerization and orchestration tools such as Docker and Kubernetes.
● Strong problem-solving skills and ability to work in a fast-paced environment.
● Excellent communication and collaboration skills, with a demonstrated ability to work effectively as part of a team.
Preferred Qualifications
● Experience in the e-commerce industry or similar high-traffic, data-intensive environments would be a big plus
● Knowledge of ETL/ELT tools like Airflow, dbt, or equivalent
● Familiarity with monitoring and observability tools for data systems (e.g., Prometheus, Grafana)