Job Description :

Senior Data Engineer

One of the most innovative and high-profile teams in News Corp is looking for a seasoned data engineer to help accelerate its vision. The role will involve combining all of News Corp’s data across categories, countries and organizations into a singular view of a customer enabling engagement, measurement and targeting. The HQData Senior Data Engineer will partner closely with product, data & engineering teams to help make this data available and accessible to our businesses, teams and partners in order to drive revenue and improve the customer experience.

Responsibilities

• Design, implement and maintain high performance big data infrastructure/systems & big data processing pipelines scaling to billions of structured and unstructured events daily

• Design, implement and maintain deep integration with up-stream systems

• Monitor performance of the data platform and optimize as needed

• Monitor and provide transparency into data quality across systems (accuracy, consistency, completeness, etc)

• Support product with the overall roadmap and ensure updates to senior leadership are 100% technically correct.

• Data analysis, understanding of business requirements and translation into logical pipelines & processes

• Conduct timely and effective research in response to specific requests (e.g. data collection, summarization, analysis, and synthesis of relevant data and information)

• Evaluate and prototype new technologies in the area of data processing

• Think quickly, communicate clearly and work collaboratively with product, data, engineering, QA and operations teams

• High energy level, strong team player and good work ethic

Technologies we use

• Google Cloud Platform (Storage, Dataflow, BigQuery, DataStudio, DataTransfer, Composer)

• Python, R, Spark (PySpark)

• Kubernetes

• Airflow

• Git for source code management

• Jira

Qualifications

• BS in Computer Science or other technical discipline

• 5+ years of experience in designing and developing big data processing systems using distributed computing

• 5+ years in Python Programming

• 5+ years experience in SQL

• 4+ years experience in Apache Airflow

• Working with REST APIs

• 5+ years of Hands on experience in Google Cloud Platform

• Expert knowledge in optimizing complex SQL queries

• Experience with job orchestration tools

• An affinity for automation

• Familiarity with networking and network application programming, including HTTP/HTTPS, JSON, and REST APIs

• Experience with at least one object oriented language (ex: Java)

• Strong OO design, data structure, and algorithm design skills

This job is posted with NTS Technology Services Pvt. Ltd.

Job Category:

News Corp is a global, diversified media and information services company focused on creating and distributing authoritative and engaging content to consumers throughout the world. The company comprises businesses across a range of media, including: news and information services, book publishing, digital real estate services, cable network programming in Australia, and pay-tv distribution in Australia.

Headquartered in New York, the activities of News Corp are conducted primarily in the United States, Australia, and the United Kingdom.