Lead Databricks Engineer Job at Programmers.io, Texas

RmgvbnhTTWlCZXptNFoyeTVLR29udXJVOGc9PQ==
  • Programmers.io
  • Texas

Job Description

One of our leading client is looking for Lead Databricks Engineer remotely.

Responsibilities:

  • Design, develop, and deploy data pipelines using Databricks, including data ingestion, transformation, and loading (ETL) processes.
  • Develop and maintain high-quality, scalable, and maintainable Databricks notebooks using Python.
  • Work with Delta Lake and other advanced features.
  • Leverage Unity Catalog for data governance, access control, and data discovery.
  • Develop and optimize data pipelines for performance and cost-effectiveness.
  • Integrate with various data sources, including but not limited to databases and cloud storage (Azure Blob Storage, ADLS, Synapse), and APIs.
  • Experience working with Parquet files for data storage and processing.
  • Experience with data integration from Azure Data Factory, Azure Data Lake, and other relevant Azure services.
  • Perform data quality checks and validation to ensure data accuracy and integrity.
  • Troubleshoot and resolve data pipeline issues effectively.
  • Collaborate with data analysts, business analysts, and business stakeholders to understand their data needs and translate them into technical solutions.
  • Participate in code reviews and contribute to best practices within the team.

Qualifications:

  • 10+ years of experience as a Data Engineer with a strong focus on Databricks.
  • Strong proficiency in Python and SQL.
  • Hands-on experience with Databricks features such as delta lake, clusters, notebooks, jobs, and workspaces.
  • Experience with data warehousing concepts and best practices.
  • Strong understanding of data modeling and data quality principles.
  • Experience with Agile development methodologies.
  • Excellent communication and collaboration skills.
  • Ability to work independently and as part of a team.

Preferred Qualifications:

  • Experience with Azure cloud services (e.g., Azure Data Factory, Azure Synapse Analytics)
  • Experience with CI/CD pipelines for Databricks workloads.
  • Experience with data visualization tools (e.g., Power BI, Tableau)
  • Experience with data security and privacy best practices

Job Tags

Remote work,

Similar Jobs

Always Best Care Senior Services - Torrance, CA

Recruiter Job at Always Best Care Senior Services - Torrance, CA

Professional Recruiter position identifies and qualifies potential in home personal attendants for older adults.Personal Attendants...  ...outgoing calls and messages Arrange 15 Interview appointments for staffing department per week Responsible for arranging 7 hires per week... 

Allied Universal

Security Guard - Front Desk Job at Allied Universal

Allied Universal, North America's leading security and facility services company, offers rewarding careers that provide you a sense of...  ...verifiable data center security experience+ 3 years of verifiable private/corporate security experience**Knowledge and Skills Required:*... 

The Jean Bradford Kline Senior Center

Kitchen Coordinator/Cook Job at The Jean Bradford Kline Senior Center

 ...cooking. Menu planning, food shopping. Must enjoy working with Seniors. Must be able to direct volunteers. Must be able to cook within...  ...budget. Must be available for all special event food prep and cooking the day of. Employment Type: Part-Time Salary: $ 15.00 Per Hour

United Health Systems

Certified Sterile Processing Tech in Exeter, NH for $38/hr Job at United Health Systems

 ...Position: Sterile Processing (SPT) Location: Exeter, NH 03833 Pay Rate: $38/hr Weekly Gross: $1,520/week Shift: Afternoons...  ...(IAHCSMM) ~2+ years current experience - Required ~ TRAVELER MUST BE CERTIFIED ~ Previous travel experience (at least... 

Joyce Windows, Sunrooms & Baths

Event-Based Appointment Setter / Paid Training & Advancement Opportunities Job at Joyce Windows, Sunrooms & Baths

 ...or Multiple Locations if applicable] Pay: Hourly + Bonuses | Weekly Pay | Potential...  ...involved, just friendly, confident, high-energy people who can generate leads like...  ...including weekends Paid training no experience needed, just the right attitude Weekly...