AWS Data Engineer Python, Dataiku Job at Compunnel, Inc., San Francisco, CA

YllVemZScWVRYzdsNHF2SW5xS2JMWFZyQWc9PQ==
  • Compunnel, Inc.
  • San Francisco, CA

Job Description

We are looking for an experienced AWS Data Engineer to design, develop, and maintain scalable data pipelines using AWS native technologies. This role involves working closely with technical analysts, stakeholders, data scientists, and other team members to ensure data quality, optimize storage solutions, and support data platform modernization efforts. The ideal candidate will have hands-on experience with cloud migration, data transformation, and automation strategies. Key Responsibilities Lead and support data platform modernization projects. Design and develop robust, scalable data pipelines using AWS native services. Optimize ETL processes for efficient data transformation and performance. Migrate workflows from on-premise environments to AWS cloud, ensuring data quality and consistency. Design automations and integrations to resolve data inconsistencies and quality issues. Perform system testing and validation to ensure successful integration and functionality. Implement security and compliance controls within the cloud environment. Conduct data validation checks pre- and post-migration to ensure completeness, consistency, and accuracy. Collaborate with data architects and developers to document manual workflows and design automation strategies. Required Qualifications Minimum of 7 years of experience in data engineering using AWS technologies (AWS Glue, Python, Snowflake, S3, Redshift). Strong experience in designing and developing scalable data pipelines. Proficiency in Snowflake for data transformations and ETL optimization. Experience with both streaming and batch data pipeline architectures. Familiarity with DataOps concepts and CI/CD pipeline setup on AWS. Hands-on experience with Databricks and a willingness to expand capabilities. Knowledge of Dataiku for data preparation and workflow automation. Strong problem-solving and analytical skills. Bachelor’s or Master’s degree in Computer Science or a related field. Preferred Qualifications Experience with AWS EMR, Lambda, and other cloud-based data engineering tools. Exposure to data governance and compliance frameworks. #J-18808-Ljbffr Compunnel, Inc.

Job Tags

Similar Jobs

Center of Excellence in Wireless Technology

Virtual Assistant Job at Center of Excellence in Wireless Technology

Role Overview: As an Entry-Level Virtual Assistant, you will provide essential administrative and operational support to our team and clients...  ..., Sheets, Calendar). Ability to work independently in a remote environment. Basic understanding of administrative and... 

Sylvan Learning Center - Minnetonka, MN

English Tutor Job at Sylvan Learning Center - Minnetonka, MN

The Sylvan Learning Centers of Minnetonka and Chaska are looking for English instructors to teach online courses or to tutor in one of our centers. Online instruction can be delivered from your location. Training can be done remotely with no need to come into the center... 

Tri- Turf Sod Farms, Inc

Semi Local Truck Driver Job at Tri- Turf Sod Farms, Inc

 ...Job Description Job Description CDL DRIVER, Tri-Turf is now taking applications...  ...Benefits: ~401(k) matching Trucking Driver Type: ~ Company driver...  ...Trucking Route: Dedicated Local Experience: ~ Driving: 3 years (Required)... 

The University of Iowa

POSTDOCTORAL SCHOLAR Job at The University of Iowa

 ...accordance with local and federal regulations, analyzing data, publishing manuscripts, and overseeing the activities of junior research...  ...year.The ideal candidate will be interested in pursuing an academic career in human neurophysiology, motor control, and/or human neuroscience... 

CXC Talent Solutions

Apartment Maintenance/Make Ready Technicians Job at CXC Talent Solutions

 ...HIRED! CXC Talent Solutions is seeking several Maintenance and/or Make-Ready technicians in Houston, TX. CXC Talent and our clients are...  ...to perform general maintenance and maintain a beautiful apartment community. Are you ready to start your new career? The ideal...