Job Information
Beacon Hill Staffing Group, LLC Lead Airflow Engineer in Saint Paul, Minnesota
*Position Description:* We are seeking a highly experienced Lead Engineer with Airflow experience to join our team in a fully remote capacity. The ideal candidate will have a strong background in development, with specific expertise in big data processing, containerization, and cloud technologies. This role will involve building and supporting pipelines and applications, working with data, and managing Kubernetes clusters. Key Responsibilities:
- Design, build, and monitor software applications and pipelines.
- Implement and maintain infrastructure on Azure with a focus on Infosec.
- Develop and manage command-line applications.
- Orchestrate processes using Kubernetes clusters.
- Utilize Synapse and MySQL databases for data management.
- Lead the tokenization project, ensuring high standards of software quality.
*Required Skills:*
- Development Experience: 5+ years
- Leadership Experience: 3+ years as a Lead, managing 3-4 Dev/QA teams
- Big Data Processing: Expertise with Apache Spark
- Workflow Orchestration: Experience with Apache Airflow/Azure Data Factory
- Programming Languages: Proficient in Python, Java, and JavaScript (React.js)
- Containerization: Hands-on experience with Kubernetes and containerization tools
- CICD: Knowledge of continuous integration and deployment practices
- Version Control: Proficiency with GitHub
- Web Development: Experience with Spring Boot and OAuth for REST APIs
- Data Visualization: Familiarity with PowerBI Cloud
- Data Management: Experience with Synapse DWH, Spark Notebooks, SQL
- ETL Tools: Awareness of tools like Pentaho DI
Beacon Hill is an Equal Opportunity Employer that values the strength diversity brings to the workplace. Individuals with Disabilities and Protected Veterans are encouraged to apply. If you would like to complete our voluntary self-identification form, please click here or copy and paste the following link into an open window in your browser: https://jobs.beaconhillstaffing.com/eeoc/ Completion of this form is voluntary and will not affect your opportunity for employment, or the terms or conditions of your employment. This form will be used for reporting purposes only and will be kept separate from all other records. Company Profile: Beacon Hill Technologies, a premier National Information Technology Staffing Group, provides world class technology talent across all industries utilizing a complete suite of staffing services. Beacon Hill Technologies' dedicated team of recruiting and staffing experts consistently delivers quality IT professionals to solve our customers' technical and business needs. Beacon Hill Technologies covers a broad spectrum of IT positions, including Project Management and Business Analysis, Programming/Development, Database, Infrastructure, Quality Assurance, Production/Support and ERP roles. Learn more about Beacon Hill Staffing Group and our specialty divisions, Beacon Hill Associates, Beacon Hill Financial, Beacon Hill HR, Beacon Hill Legal, Beacon Hill Life Sciences and Beacon Hill Technologies by visiting www.beaconhillstaffing.com . We look forward to working with you. Beacon Hill. Employing the Future™