Web3 Data Engineer
April 12
🏡 Remote – New York
Description
• Develop data pipelines to collect and process blockchain data • Extract raw data from various data sources • Build data pipelines to process and load data • Manage cloud infrastructure for data pipeline operation • Develop CI/CD and monitoring system for building automated pipelines • Build a data warehouse for blockchain data analysis • Develop blockchain domain data mart data quality management • Build data pipeline of Bitcoin / EVM (e.g. Ethereum) / Non-EVM series blockchains • Develop blockchain data analysis data mart • Develop data API for serving blockchain data
Requirements
• Fits well with CryptoQuant culture • Experience with data processing using Python, SQL • Experience with workflow management platforms (e.g. Apache Airflow) • Experience building and operating data marts for various purposes • Experience with Docker Containers or Kubernetes • Experience in building CI / CD • Experience with Git / GitHub • Experience with Python web frameworks (e.g. Flask, FastAPI) Nice to have: • Experience in designing, building, and operating real-time data processing (e.g. Kafka) or micro-batch data processing pipelines in a lead role • Experience building, developing, and operating Kubernetes clusters at work • Experience working with data warehousing solutions (e.g. Hadoop, Spark) • Experience working with data discovery platforms (e.g DataHub, Amundsen) • At least 3 years of relevant experience with RDB, NoSQL, etc. • Experience in using and managing cloud services (e.g. AWS, GCP) / data platform (e.g. Snowflake) • Knowledge of various blockchain concepts and structures • Experience with blockchain node (e.g. Bitcoin, Ethereum) operations or on-chain data • Strong communication skills and good written and spoken English ability
Benefits
• Gain experience in operating large-scale data warehouses • Develop TB-scale or greater web3 data • Participate in design and implementation of large-scale parallel processing and efficient workflow design • Collaborate with global leading web3 data scientists and analysts • Actively execute projects with project ownership
Apply Now
