Big Data Hadoop Architect Hybrid
Job Description:
We are hiring a Big Data Hadoop Architect to lead the design and modernization of enterprise-scale data platforms. This role requires a hands-on architect who can balance strategy with execution, drive cloud transformation, and build highly scalable, cost-efficient data ecosystems.
Key Responsibilities (Expanded & Realistic)
Define enterprise data architecture roadmap aligned with business goals
Architect and implement large-scale Hadoop/Spark-based data platforms
Lead migration from on-prem Hadoop clusters to cloud-native architectures
Design real-time and batch data pipelines using Kafka, Spark Streaming, Flink, etc.
Establish data lake / lakehouse architecture (Delta Lake, Iceberg, Hudi)
Drive data modeling standards (dimensional, data vault, normalized models)
Build frameworks for data ingestion (ETL/ELT) from structured & unstructured sources
Optimize cluster performance, cost, and resource utilization
Define and enforce data governance, lineage, cataloging, and quality frameworks
Implement security best practices (encryption, IAM, data masking, compliance – HIPAA/GDPR where applicable)
Lead capacity planning and scalability strategies for petabyte-scale data
Evaluate and introduce modern tools (Databricks, Snowflake, etc.) beyond Hadoop ecosystem
Work closely with DevOps teams to implement CI/CD pipelines for data workflows
Provide technical leadership, code reviews, and mentoring to engineering teams
Collaborate with business stakeholders to translate requirements into technical solutions
Drive incident resolution and root cause analysis for production data issues
Ensure high availability and disaster recovery strategies are in place
Influence tooling, vendor selection, and long-term data platform investments
Required Skills & Experience
10–15+ years of IT experience with minimum 5+ years in Big Data Architecture
Strong expertise in:
Hadoop Ecosystem (HDFS, YARN, Hive, HBase)
Apache Spark (Core, SQL, Streaming)
Kafka / Pulsar / Flink
Hands-on experience with cloud platforms:
AWS (EMR, S3, Glue, Redshift) OR
Azure (Data Lake, Synapse, HDInsight) OR
GCP (Dataproc, BigQuery)
Strong programming skills in Java / Scala / Python
Deep understanding of distributed systems, parallel processing, and data partitioning
Experience with containerization (Docker, Kubernetes)
Expertise in data warehousing and ETL frameworks
Preferred Qualifications
Experience with Databricks, Snowflake, or modern lakehouse platforms
Exposure to AI/ML pipelines and feature engineering systems
Cloud certifications (AWS/Azure/GCP Architect level)
Strong leadership and stakeholder management experience
Total Rewards & Benefits (Realistic US Package)
Comprehensive Medical, Dental & Vision Insurance
401(k) with employer match (up to 5–6%)
Paid Time Off (20–25 days) + Paid Holidays
Parental Leave (12–16 weeks)
Learning Budget ($2,500/year for certifications & courses)
Home Office Setup Allowance ($1,000 one-time)
Wellness Programs (Gym reimbursement, mental health support)
Required Skills:
Encryption Containerization Performance CI/CD pipelines Hipaa Data Databricks Distributed Systems Cloud Data Governance Modeling Support High Availability Balance Hive ROOT Scala Disaster Recovery ETL Data Warehousing Apache Spark Investments Pipelines Spark Big Data Root Cause Analysis Hadoop Apache Kafka Gcp Hiring Analysis Snowflake Scalability Data Modeling Compliance Reviews Mentoring CI/CD Stakeholder Management Azure Architecture Availability Insurance DevOps Programming Strategy AWS Kubernetes Security Docker Planning Design Engineering Business Java SQL Python Leadership Management