Posted 3w ago

Principal Data Engineer

@ Applied Information Sciences
Hyderabad, Telangana, India
HybridFull Time
Responsibilities:design pipelines, lead architecture, mentor engineers
Requirements Summary:8+ years in data or platform engineering; cloud data pipelines on Azure/AWS/GCP; strong SQL, Python; Databricks/Lakehouse; data models, governance and security; experience with ingestion tools.
Technical Tools Mentioned:Databricks, Delta Lake, Spark, Python, SQL, Airflow, Matillion, FiveTran, Airbyte
Save
Mark Applied
Hide Job
Report & Hide
Job Description

Why AIS?

When you join AIS, you’re joining a mission-driven team that’s passionate about making a difference. You’ll work on projects that matter, alongside industry-leading experts, in an environment that fosters innovation, driving client success, and empowering our team to make a lasting impact. As an employee-owned company, we value collaboration, inclusivity, continuous growth, and shared success.

  • Employee Ownership: Your contributions directly impact the company’s success, and you share in its achievements.

  • Continuous Learning: Access to resources, training, and mentorship to support your professional growth.

  • Inclusive Culture: A workplace where diversity is celebrated, and everyone’s voice is valued.

  • Mission-Driven Work: Engage in projects that make a meaningful difference for our clients and communities.

 

What are we looking for?

At AIS, we're looking for more than just skills - we're looking for driven individuals who are passionate about making a difference, eager to grow, and aligned with our core principles.

 

What you will be doing?

 

This position is contingent upon contract award. We are currently pursuing a proposal and are seeking qualified candidates to include in our submission and identify candidates for future hiring needs on the program once awarded.

At AIS, we are dedicated to providing our employees with diverse opportunities to grow their careers while supporting a variety of impactful projects. For this position, we are seeking a talented individual to join AIS as a Lead Data Analytics Engineer.
  • Core Knowledge & Skills: Masters SQL and data architecture; builds end-to-end pipelines on cloud platforms; ensures security and compliance; leverages APIs and advanced AI/ML; plans and leads projects and team resources.

  • Work & Complexity: Owns full lifecycle delivery; resolves ambiguous, enterprise-scale problems; shapes technical direction and project outcomes.

  • Quality & Independence: Establishes best practices and performance metrics; oversees QA for major initiatives; operates with wide decision latitude.

  • Teamwork & Communication: Leads teams implementing complex solutions; aligns with business objectives; serves as primary liaison to executives; drives structured feedback.

  • Consulting & Engagement: Advises executive stakeholders on strategic data initiatives; leads consulting teams; embeds innovative, data-driven

As your initial project assignment, you will support the unique needs of our client as a Principal Data Engineer.

Project Summary

The Principal Data Engineer is a senior, hands‑on technical leader responsible for designing, building, and evolving scalable, reliable, and cost‑efficient data and analytic platforms that power analytics, reporting, and advanced AI/predictive use cases. This role sets engineering standards, drives platform modernization, and mentors engineering teams while partnering closely with architecture, analytics, product, and governance stakeholders.

Key Responsibilities

Technical Leadership & Architecture

  • Lead the design and evolution of enterprise‑scale data platforms, including ingestion, transformation, storage, and consumption layers
  • Define and enforce data engineering standards, patterns, and best practices (e.g., pipeline design, testing, CI/CD, observability)
  • Serve as a technical authority on data architecture decisions, trade‑offs, and platform strategy.

Platform & Pipeline Engineering

  • Design, build, and optimize high‑performance batch and streaming data pipelines
  • Improve reliability, performance, scalability, and cost efficiency of existing data assets
  • Implement reusable frameworks and reference architectures to accelerate delivery across teams
  • You apply subject matter knowledge and skills across multiple areas to assess the impact of complex issues and implement long-term solutions. You foster innovation using subject matter knowledge to enhance tools, practices, and processes for the team.

Databricks & Lakehouse Focus (Strongly Preferred)

  • Design and operate Databricks‑based Lakehouse solutions using Delta Lake, Spark, and notebooks/jobs
  • Establish best practices for Databricks workspace architecture, security, performance tuning, and cost management
  • Lead or support migrations to Databricks and modernization of legacy data pipelines

Data Quality, Governance & Security

  • Partner with data governance and security teams to embed data quality checks, metadata, lineage, and access controls into pipelines
  • Ensure platforms support regulatory, compliance, and enterprise risk requirements

Cross‑Functional Collaboration

  • Work closely with data architects, analytic engineers, data scientists, product owners, and business stakeholders
  • Translate business and analytic requirements into scalable technical solutions
  • Influence roadmaps by tying platform investments to measurable business outcomes

Mentorship & Engineering Excellence

  • Mentor senior and mid‑level engineers; raise the overall technical bar of the team
  • Lead by example through hands‑on contribution to critical pipelines and platform components

Required For This Opportunity

  • University or college degree in Computer Science, Information Systems, Mathematics, Engineering, Statistics, or related field of study.
  • 8+ years of experience in data engineering or platform engineering roles
  • Deep expertise in building large‑scale data pipelines and platforms in cloud environments (Azure, AWS, or GCP)
  • Strong proficiency with distributed data processing (e.g., Spark)
  • Advanced SQL and strong Python programming skills
  • Experience designing data models and architectures for analytics and downstream consumption
  • Experience in data ingestion tools such as Matillion, FiveTran, Airbyte, etc. a plus
  • Strong analytical skills and experience with data modelling
  • Strong project management and organizational skills
  • Proven ability to work independently and with a team
  • Extensive hands‑on experience with Databricks and Lakehouse architecture strongly preferred
  • Experience with Delta Lake, Spark optimization, and cost/performance tuning strongly preferred
  • Experience supporting advanced analytics, machine learning, or AI‑driven use cases
  • Familiarity with modern data tooling (e.g., orchestration, data quality frameworks, metadata/catalog tools)
  • Prior experience setting engineering standards or leading platform modernization initiatives

Nice to Have Skills

Technical competencies

Subject Matter Expertise

  • Developing expertise
  • You strengthen your depth and/or breadth of subject matter knowledge and skills across multiple areas.
  • You define the expertise required in your area based on emerging technologies, industry practices. You build the team’s capability accordingly.
  • Applying expertise

Solution Development

  • Systems thinking
  • You lead and foster collaboration across our Data Platform Technology to develop solutions to complex issues.
  • You apply a whole systems approach to evaluating impact, and take ownership for ensuring links between structure, people and processes are made.
  • Focusing on quality
  • You instill a quality mindset to the team and ensure the appropriate methods, processes and standards are in place for teams to deliver quality solutions. You create and deliver improvement initiatives.

Technical Communication

  • Simplifying complexity
  • You develop tools, aids and/or original content to support the delivery and/or understanding of complex information. You guide others on best practice.
  • Influencing stakeholders
  • You are adaptive across a range of situations, effectively tailoring your delivery methods based on your knowledge of stakeholders to have maximum impact and influence.
At AIS, we are committed to offering competitive and fair compensation that reflects the skills, experience, and contributions of each team member. The targeted base salary range for this role is $0-$0 per year. Please note that this range is provided as a guideline and the final offer will be based on several factors, including but not limited to, skillset and competencies, level of experience, education, certifications, and location. We value transparency in our hiring process and are happy to discuss how your unique qualifications align with our compensation structure during the interview process.

Applied Information Sciences does not discriminate on the basis of race, national origin, religion, color, gender, sexual orientation, age, disability, protected veteran status, or any other basis. Employment decisions are based solely on qualifications, merit, and business needs.