Senior Data Platform Engineer

Key Responsibilities:

• Collaborate with Solution Architects and Data Engineers to implement scalable and efficient data architectures. Ensure alignment with industry best practices and emerging technologies.
• Manage infrastructure components across on-premises and cloud environments including provisioning, scaling, and governance.
• Implement data governance, security, and compliance policies.
• Implement strategies for data retention, archival, and recovery.
• Monitor system performance and implement scaling strategies, ensure high availability and disaster recovery capabilities.
• Provide frameworks and tooling for data management, data profiling and quality.
• Standardize data access methods to improve efficiency and security.
• Monitor and optimize cloud resource utilization.
• Design and implement monitoring solutions for proactive issue detection.
• Implement workflow automation.

Skills & Qualifications:

Technical Skills:

• Cloud Platforms: Expertise in AWS, Azure, or Google Cloud Platform.
• Infrastructure as Code (IaC): Proficiency with Terraform, CloudFormation, or similar tools.
• Cluster Management: Automation, cost optimization, cluster scaling.
• Monitoring & Governance: AWS CloudWatch, Databricks workspace monitoring.
• Containerization and Orchestration: Experience with Docker, Kubernetes.
• Data Storage Technologies: Knowledge of databases, data warehouses and data lakes.
• Monitoring Tools: Familiarity with tools like Prometheus, Grafana, or CloudWatch.
• Automation Scripting: Ability to write scripts in Python, Bash, or PowerShell.
• Security and Compliance: Understanding of IAM, encryption, network security, and compliance standards.

Soft Skills:

• Problem-Solving: Ability to diagnose complex issues and design effective solutions.
• Collaboration: Strong teamwork skills to work across various teams and departments.
• Communication: Clear verbal and written communication for documentation and stakeholder interactions.
• Adaptability: Willingness to learn new technologies and adapt to changing project requirements.

Experience:

• Education: Bachelor's or Master's degree in Computer Science, Engineering, or related field.
• Professional Experience: At least 5 years in platform engineering, DevOps, or related roles.
• Project Experience: Experience with large-scale data platforms or infrastructure projects.