JOB SUMMARY: We are seeking a skilled Data Engineer to join our Analytics & Automation team and serve as the primary architect and builder of our data pipelines. This role will be responsible for designing, implementing, and maintaining scalable, cost-effective data solutions that power enterprise analytics and reporting. The ideal candidate will work closely with business stakeholders to identify data needs, ensure high data quality, and optimize our use of Azure-based technologies for ongoing innovation and performance.
ESSENTIAL DUTIES
· Design, build, and maintain robust, scalable data pipelines in Azure using services such as Azure Data Factory, Azure Synapse, and Azure Data Lake.
· Ingest and integrate data from diverse sources (APIs, databases, files, etc.) into centralized repositories for analytics use.
· Develop and manage ETL/ELT workflows that transform raw data into clean, analytics-ready models aligned with business requirements.
· Create microservices architecture that rapidly scales with business growth and AI capabilities.
· Build data pipelines that feed AI models with clean, structured logistics data.
· Implement real-time analytics engines powering predictive logistics capabilities.
· Ensure data quality, integrity, and governance through validation, monitoring, and alerting systems.
· Collaborate with business and technical teams to gather data requirements and ensure alignment between data assets and business needs.
· Implement cost-conscious data engineering strategies, optimizing performance while managing cloud consumption and storage expenses.
· Maintain and enhance a semantic data layer or curated datasets to support self-service BI (Power BI) and advanced analytics.
· Contribute to the architecture of our data platform, including security, scalability, and performance best practices.
· Automate data operations and deployment processes using CI/CD pipelines and infrastructure-as-code (e.g., ARM, Bicep, or Terraform).
· Stay current with emerging Azure technologies and recommend improvements to existing data workflows
· Other duties as assigned.