divspan id=docs-internal-guid-0dd4f100-7fff-ac94-a599-a105d6031434p dir=ltr style=line-height: 1.38; margin-bottom: 12pt; margin-top: 12pt;span style=font-family: Arial, sans-serif; font-size: 11pt; vertical-align: baseline;Bloomberg runs on data. Its our business and our product. From the biggest banks to elite hedge funds, financial institutions need timely, accurate data to capture opportunities and evaluate risk in fast-moving markets. With petabytes of data available, a solution to transform and analyze the data is critical to our success./span/pp dir=ltr style=line-height: 1.38; margin-bottom: 12pt; margin-top: 12pt;span style=font-family: Arial, sans-serif; font-size: 11pt; vertical-align: baseline;Bloomberg’s Data Science Platform was established to support development efforts around data-driven science, machine learning, and business analytics. The solutionnbsp; aims to provide scalable compute, specialized hardware and first-class support for a variety of workloads such as ML training job and inference services, Spark, and Jupyter. The solution was developed to provide a standard set of tooling for addressing the Model Development Life Cycle from experimentation and training to inference. The solution is built using containerization, container orchestration and cloud architecture and built on top of 100% open source foundations./span/pp dir=ltr style=line-height: 1.38; margin-bottom: 12pt; margin-top: 12pt;span style=font-family: Arial, sans-serif; font-size: 11pt; vertical-align: baseline;Production Inference is a critical step on the MDLC to realize the business value for Bloomberg AI applications and the advent of large language models (LLMs) presents new opportunities for expanding NLP capabilities in our products. The inference solution is powered by open source project KServe which is a production ready inference solution for both generative and predictive AI applications. We are poised for enormous user growth this year and have an ambitious roadmap in terms of new features as well as improved user experience. That’s where you come in. As a member of the inference team, you’ll have the opportunity to design and implement scalable, low latency, high throughput model inference solutions in a hybrid cloud environment. We are founding members of the KServe project to standardize ML Inference within the Kubernetes ecosystem. As part of that, we regularly upstream features we develop, present at conferences and collaborate with our peers in the industry. Open source is at the heart of our team. Its not just something we do in our free time, it is how we work./span/ph2 dir=ltr style=line-height: 1.38; margin-bottom: 4pt; margin-top: 18pt;span style=font-family: Arial, sans-serif; font-size: 17pt; vertical-align: baseline;We’ll trust you to:/span/h2ul style=margin-bottom: 0px; margin-top: 0px;li dir=ltr style=font-family: Arial, sans-serif; font-size: 11pt; list-style-type: disc; vertical-align: baseline; white-space: pre;p dir=ltr style=line-height: 1.38; margin-bottom: 0pt; margin-top: 12pt;span style=font-size: 11pt; vertical-align: baseline;Interact with data scientists to understand their production use cases and requirements to advise the next set of GenAI features for the inference platform./span/p/lili dir=ltr style=font-family: Arial, sans-serif; font-size: 11pt; list-style-type: disc; vertical-align: baseline; white-space: pre;p dir=ltr style=line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;span style=font-size: 11pt; vertical-align: baseline;Design solutions for problems such as scalable model deployment, low latency/high throughput inference, GPU resource optimizations and autoscaling./span/p/lili dir=ltr style=font-family: Arial, sans-serif; font-size: 11pt; list-style-type: disc; vertical-align: baseline; white-space: pre;p dir=ltr style=line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;span style=font-size: 11pt; vertical-align: baseline;Automate operation and improve telemetry of the inference platform in our infrastructure stack./span/p/lili dir=ltr style=font-family: Arial, sans-serif; font-size: 11pt; list-style-type: disc; vertical-align: baseline; white-space: pre;p dir=ltr style=line-height: 1.38; margin-bottom: 12pt; margin-top: 0pt;span style=font-size: 11pt; vertical-align: baseline;Design solutions for multi-cloud strategy./span/p/li/ulh2 dir=ltr style=line-height: 1.38; margin-bottom: 4pt; margin-top: 18pt;span style=font-family: Arial, sans-serif; font-size: 17pt; vertical-align: baseline;You’ll need to be able to:/span/h2ul style=margin-bottom: 0px; margin-top: 0px;li dir=ltr style=font-family: Arial, sans-serif; font-size: 11pt; list-style-type: disc; vertical-align: baseline; white-space: pre;p dir=ltr style=line-height: 1.38; margin-bottom: 0pt; margin-top: 12pt;span style=font-size: 11pt; vertical-align: baseline;Innovate and design solutions that keep in mind strict production SLA: low latency/high throughput, multi-tenancy, high availability, reliability across clusters/data centers, etc./span/p/lili dir=ltr style=font-family: Arial, sans-serif; font-size: 11pt; list-style-type: disc; vertical-align: baseline; white-space: pre;p dir=ltr style=line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;span style=font-size: 11pt; vertical-align: baseline;Fix and optimize generative inference application performance./span/p/lili dir=ltr style=font-family: Arial, sans-serif; font-size: 11pt; list-style-type: disc; vertical-align: baseline; white-space: pre;p dir=ltr style=line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;span style=font-size: 11pt; vertical-align: baseline;Provide developer and operational documentation./span/p/lili dir=ltr style=font-family: Arial, sans-serif; font-size: 11pt; list-style-type: disc; vertical-align: baseline; white-space: pre;p dir=ltr style=line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;span style=font-size: 11pt; vertical-align: baseline;Provide performance analysis and capacity planning for clusters./span/p/lili dir=ltr style=font-family: Arial, sans-serif; font-size: 11pt; list-style-type: disc; vertical-align: baseline; white-space: pre;p dir=ltr style=line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;span style=font-size: 11pt; vertical-align: baseline;String communication and collaboration skills, with the ability to work effective with multi-functional teams/span/p/lili dir=ltr style=font-family: Arial, sans-serif; font-size: 11pt; list-style-type: disc; vertical-align: baseline; white-space: pre;p dir=ltr style=line-height: 1.38; margin-bottom: 12pt; margin-top: 0pt;span style=font-size: 11pt; vertical-align: baseline;Have a passion for providing reliable and scalable infrastructure./span/p/li/ulh2 dir=ltr style=line-height: 1.38; margin-bottom: 4pt; margin-top: 18pt;span style=font-family: Arial, sans-serif; font-size: 17pt; vertical-align: baseline;You’ll need to have:/span/h2ul style=margin-bottom: 0px; margin-top: 0px;li dir=ltr style=font-family: Arial, sans-serif; font-size: 11pt; list-style-type: disc; vertical-align: baseline; white-space: pre;p dir=ltr style=line-height: 1.38; margin-bottom: 0pt; margin-top: 12pt;span style=font-size: 11pt; vertical-align: baseline;4+ years programming experience in two or more languages (e.g., Python, Go, C++)/span/p/lili dir=ltr style=font-family: Arial, sans-serif; font-size: 11pt; list-style-type: disc; vertical-align: baseline; white-space: pre;p dir=ltr style=line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;span style=font-size: 11pt; vertical-align: baseline;A Degree in Computer Science, Engineering or similar field of study or equivalent work experience/span/p/lili dir=ltr style=font-family: Arial, sans-serif; font-size: 11pt; list-style-type: disc; vertical-align: baseline; white-space: pre;p dir=ltr style=line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;span style=font-size: 11pt; vertical-align: baseline;Experience designing and implementing low-latency, high-scalability inference platform./span/p/lili dir=ltr style=font-family: Arial, sans-serif; font-size: 11pt; list-style-type: disc; vertical-align: baseline; white-space: pre;p dir=ltr style=line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;span style=font-size: 11pt; vertical-align: baseline;Design, develop, test and deploy inference solutions for LLMs/span/p/lili dir=ltr style=font-family: Arial, sans-serif; font-size: 11pt; list-style-type: disc; vertical-align: baseline; white-space: pre;p dir=ltr style=line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;span style=font-size: 11pt; vertical-align: baseline;Explore emerging inference optimization techniques/span/p/lili dir=ltr style=font-family: Arial, sans-serif; font-size: 11pt; list-style-type: disc; vertical-align: baseline; white-space: pre;p dir=ltr style=line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;span style=font-size: 11pt; vertical-align: baseline;Experience with debugging performance issues with distributed tracing./span/p/lili dir=ltr style=font-family: Arial, sans-serif; font-size: 11pt; list-style-type: disc; vertical-align: baseline; white-space: pre;p dir=ltr style=line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;span style=font-size: 11pt; vertical-align: baseline;Experience working with a distributed multi-tenancy and multi-cluster system./span/p/lili dir=ltr style=font-family: Arial, sans-serif; font-size: 11pt; list-style-type: disc; vertical-align: baseline; white-space: pre;p dir=ltr style=line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;span style=font-size: 11pt; vertical-align: baseline;Experience with distributed systems eg. Kubernetes, Kafka, RabbitMQ, Zookeeper/Etcd./span/p/lili dir=ltr style=font-family: Arial, sans-serif; font-size: 11pt; list-style-type: disc; vertical-align: baseline; white-space: pre;p dir=ltr style=line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;span style=font-size: 11pt; vertical-align: baseline;Strong knowledge of data structures and algorithms./span/p/lili dir=ltr style=font-family: Arial, sans-serif; font-size: 11pt; list-style-type: disc; vertical-align: baseline; white-space: pre;p dir=ltr style=line-height: 1.38; margin-bottom: 12pt; margin-top: 0pt;span style=font-size: 11pt; vertical-align: baseline;Linux systems experience (Network, OS, Filesystems)./span/p/li/ulh2 dir=ltr style=line-height: 1.38; margin-bottom: 4pt; margin-top: 18pt;span style=font-family: Arial, sans-serif; font-size: 17pt; vertical-align: baseline;We’d love to see:/span/h2ul style=margin-bottom: 0px; margin-top: 0px;li dir=ltr style=font-family: Arial, sans-serif; font-size: 11pt; list-style-type: disc; vertical-align: baseline; white-space: pre;p dir=ltr style=line-height: 1.38; margin-bottom: 0pt; margin-top: 12pt;span style=font-size: 11pt; vertical-align: baseline;Experience Large Language Model Inference, especially vLLM, TensorRT-LLM runtimes./span/p/lili dir=ltr style=font-family: Arial, sans-serif; font-size: 11pt; list-style-type: disc; vertical-align: baseline; white-space: pre;p dir=ltr style=line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;span style=font-size: 11pt; vertical-align: baseline;Experience with Kubeflow/KServe, MLFlow, Sagemaker./span/p/lili dir=ltr style=font-family: Arial, sans-serif; font-size: 11pt; list-style-type: disc; vertical-align: baseline; white-space: pre;p dir=ltr style=line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;span style=font-size: 11pt; vertical-align: baseline;Experience working with GPU compute software and hardware./span/p/lili dir=ltr style=font-family: Arial, sans-serif; font-size: 11pt; list-style-type: disc; vertical-align: baseline; white-space: pre;p dir=ltr style=line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;span style=font-size: 11pt; vertical-align: baseline;Ability to identify and perform OS and hardware-level optimizations./span/p/lili dir=ltr style=font-family: Arial, sans-serif; font-size: 11pt; list-style-type: disc; vertical-align: baseline; white-space: pre;p dir=ltr style=line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;span style=font-size: 11pt; vertical-align: baseline;Open source involvement such as a well-curated blog, accepted contribution, or community presence./span/p/lili dir=ltr style=font-family: Arial, sans-serif; font-size: 11pt; list-style-type: disc; vertical-align: baseline; white-space: pre;p dir=ltr style=line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;span style=font-size: 11pt; vertical-align: baseline;Experience with cloud LLM providers such as AWS Redrock, Gemini or Azure OpenAI./span/p/lili dir=ltr style=font-family: Arial, sans-serif; font-size: 11pt; list-style-type: disc; vertical-align: baseline; white-space: pre;p dir=ltr style=line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;span style=font-size: 11pt; vertical-align: baseline;Experience with configuration management systems (Terraform, Ansible)/span/p/lili dir=ltr style=font-family: Arial, sans-serif; font-size: 11pt; list-style-type: disc; vertical-align: baseline; white-space: pre;p dir=ltr style=line-height: 1.38; margin-bottom: 12pt; margin-top: 0pt;span style=font-size: 11pt; vertical-align: baseline;Experience with continuous integration tools and technologies (Jenkins, Git, Chat-ops)/span/p/li/ulh2 dir=ltr style=line-height: 1.38; margin-bottom: 4pt; margin-top: 18pt;span style=font-family: Arial, sans-serif; font-size: 17pt; vertical-align: baseline;Learn more about our work using the links below:/span/h2ul style=margin-bottom: 0px; margin-top: 0px;li dir=ltr style=font-family: Arial, sans-serif; font-size: 11pt; list-style-type: disc; vertical-align: baseline; white-space: pre;p dir=ltr style=line-height: 1.38; margin-bottom: 0pt; margin-top: 12pt;span style=font-size: 11pt; vertical-align: baseline;Keynote: Platform Building Blocks: How to build ML infrastructure with CNCF projects -/spana href=https://www.youtube.com/watch?v=ncED2EMcxZ8span style=font-size: 11pt; color: rgb(0, 0, 0); vertical-align: baseline; /spanspan style=font-size: 11pt; color: rgb(17, 85, 204); vertical-align: baseline;https://www.youtube.com/watch?v=ncED2EMcxZ8/span/a/p/lili dir=ltr style=font-family: Arial, sans-serif; font-size: 11pt; list-style-type: disc; vertical-align: baseline; white-space: pre;p dir=ltr style=line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;span style=font-size: 11pt; vertical-align: baseline;The State and Future of Cloud Native Model Inference -/span/p/lili dir=ltr style=font-family: Arial, sans-serif; font-size: 11pt; list-style-type: disc; vertical-align: baseline; white-space: pre;p dir=ltr style=line-height: 1.38; margin-bottom: 0pt; margin-top: 0pt;a href=https://www.youtube.com/watch?v=786VaGAfm6Ispan style=font-size: 11pt; color: rgb(17, 85, 204); vertical-align: baseline;https://www.youtube.com/watch?v=786VaGAfm6I/span/a/p/lili dir=ltr style=font-family: Arial, sans-serif; font-size: 11pt; list-style-type: disc; vertical-align: baseline; white-space: pre;p dir=ltr style=line-height: 1.38; margin-bottom: 12pt; margin-top: 0pt;span style=font-size: 11pt; vertical-align: baseline;The Hitchhikers Guide to Kubernetes Platforms: Don’t Panic, Just Launch!/spana href=https://www.youtube.com/watch?v=a84mwXicpdcspan style=font-size: 11pt; color: rgb(0, 0, 0); vertical-align: baseline; /spanspan style=font-size: 11pt; color: rgb(17, 85, 204); vertical-align: baseline;https://www.youtube.com/watch?v=a84mwXicpdc/span/a/p/li/ulbr/span/div

Salary: 160000,240000,USD,Annual

Bloomberg is an equal opportunity employer and we value diversity at our company. We do not discriminate on the basis of age, ancestry, color, gender identity or expression, genetic predisposition or carrier status, marital status, national or ethnic origin, race, religion or belief, sex, sexual orientation, sexual and other reproductive health decisions, parental or caring status, physical or mental disability, pregnancy or parental leave, protected veteran status, status as a victim of domestic violence, or any other classification protected by applicable law.

Bloomberg is a disability inclusive employer. Please let us know if you require any reasonable adjustments to be made for the recruitment process. If you would prefer to discuss this confidentially, please email amer_recruit@bloomberg.net