Senior Data Engineer

Senior Data Engineer at Mechanon Senior Data Engineer MECHANON Bangalore, India (On-site) Compensation: £12,000–£16,000 per year (GBP-anchored) + 0.25% Venture Equity + Company Performance Bonus About MECHANON: Mechanon is a Dubai-based venture studio. We build businesses from concept, launch and scale. We build real businesses—not agency projects or corporate innovation experiments. Our current venture, The Engine, is building a production intelligence platform for the global film and television industry. The Role: We are hiring a Senior Data Engineer to build the data acquisition and ingestion platform that powers The Engine. Your primary challenge is collecting, cleaning, validating, deduplicating and continuously updating large-scale, messy, real-world data before AI ever touches it. The platform is only as good as the data underneath it. Your work is the moat. This role is approximately 70% data engineering and 30% applied AI. You will work closely with the Founder and Platform Engineering Lead to design systems that acquire, structure, enrich, resolve and operationalise real-world industry data at scale. This is a hands-on building role from day one. Your Core Work (Data Engineering): • Design and operate large-scale ingestion pipelines that continuously collect, validate, deduplicate and update millions of real-world records from unreliable external sources. • Data acquisition from RSS feeds, public datasets, REST APIs, websites and authenticated web applications. • End-to-end orchestration of reliable data pipelines. • Data cleaning, transformation and modelling (ETL/ELT). • Layered data model: immutable raw observations → staging → confidence-scored canonical records → application schema. • Schema design, data quality, validation and integrity. • Entity resolution and relationship graph infrastructure with full audit trail. • APIs and data services feeding the wider platform. On Top of the Data Layer (AI) • AI-assisted extraction, classification and enrichment. • Confidence scoring and validation of AI output. • Knowing when deterministic rules are better than AI. Technology Stack Python • PostgreSQL (AWS RDS) • Prefect • dbt • Playwright • BeautifulSoup • feedparser • AWS Fargate • OpenAI / Claude APIs • PostgreSQL relationship graph (Neo4j planned) Hard Requirements: • Real experience building data acquisition and ingestion systems from messy external sources. • Advanced Python. • Strong SQL. • PostgreSQL. • Pipeline orchestration (Prefect, Airflow, Dagster or similar). • ETL/ELT and data modelling. • Data quality, validation and idempotent pipeline design. • APIs and production data services. • Experience building systems that remain reliable as sources change. Highly Desirable • Playwright or browser automation. • Entity resolution and deduplication. • Knowledge graphs. • Incremental loading, backfills and late-arriving data. • Monitoring for silent data failures. • dbt. • OpenAI/Claude APIs in production. We're Looking For People Who Already • Built and owned production data systems end to end. • Worked fluently with messy real-world data. • Built systems that collect data from the real world, not just analyse data that already existed. • Built pipelines that recover cleanly when sources fail. • Automated repetitive work and shipped systems people rely on. We're Not Looking For: • Prompt-only AI specialists. • No-code operators. • Dashboard/reporting-only data engineers. • Theory-first demo builders. • Engineers who require heavy process or constant supervision. The Kind of Person We're Looking For: Builder mentality • Low ego • Systems thinker • Fast learner • Comfortable with ambiguity • Autonomous • Execution-focused • Strong discipline around data quality. If you're a strong data engineer who is lighter on AI but excited to learn, we still want to hear from you. Data acquisition and ingestion is the hardest part of this role—and the foundation of everything else.