What You Will Do

The High Performance Computing environments group (HPC-ENV) is seeking a driven applicant for the HPC Programming & Runtime Environments (PRE) Team. This team has two primary responsibilities. Foremost, it is responsible for HPC software installation and product maintenance, support for third-party packages, providing software related network services to the HPC clusters, and vendor communication for supported software products. The PRE Team is also responsible for ensuring the usability of HPC systems at LANL via thorough testing and maintains both a large suite of such tests and the open-source testing framework through which those tests are performed.

The HPC Division supports the Los Alamos National Laboratory (LANL) mission by managing a world-class supercomputing center. We support stockpile stewardship for NNSA/DOE and accelerate scientific discovery for scientists. We integrate and support some of the world's largest supercomputers during an exciting time in computing with the focus on traditional large scale simulations, data science, artificial intelligence, and machine learning.

HPC-ENV manages how users interact with the HPC systems at LANL which helps secure the nation and pushes the boundaries of science and innovation. Several teams within HPC-ENV are responsible for the broad range of HPC platforms, programming and runtime environments, software, application optimization and readiness, software engineering, user support & services for a large and diverse customer base. We provide support and services to many production platforms at a world-class computing facility to ensure customers can accomplish their research and mission at extreme scale.

This position will be filled at either the Scientist 2 or Scientist 3 level, depending on the skills of the selected candidate. Additional job responsibilities (outlined below) will be assigned if the candidate is hired at the higher level.

Scientist 2 ($101,7000 - $168,200)

The successful candidate will perform the full spectrum of tasks, including but not limited to:

• Install, support and maintain software used on HPC resources at the Los Alamos National Laboratory, largely using Spack.

• Work with Spack developers to add new features and fix bugs as needed.

• Work across organizational HPC groups on tool installation, management, and testing.

• Provide support for AI/ML tools on GPU based clusters.

• Research, evaluate and recommend new software tools and packages for usage in LANL's HPC software stack

• Work closely with software vendors and provide support for third party packages

• Contribute to the development of novel custom tools, especially in the areas of software deployment, and testing

• Interact with end-users and the HPC user support teams

• Provide tool education opportunities to users of LANL's HPC platforms

• Participate in a weekly on-call rotation to provide production support coverage

• Contributes to the development of technical presentations, papers, technical reports, etc. Communicates results through appropriate channels.

• May need to work under pressure of frequent interruptions and conflicting priorities.

Scientist 3 ($122,300 - $206,300)

In addition to the Job Requirements outlined above, qualification at the Scientist 3 level requires:

• Technical lead within the Programming & Runtime Environments team at least 1 or more areas

• Provide technical direction and help to continuously improve the state of HPC software support at LANL

• Represent the Programming & Runtime Environments Team at workshops, conferences, and meetings with other HPC sites

• Represent LANL HPC across the DOE Complex

• Use project assignments to further organizational goals

• Leads peer review of the work of others within the organization

• Enhance technical and professional expertise of junior staff through mentoring and training

• Actively participates in NNSA/DOE interactions, special assignments, and/or external collaborations including the NNSA/DOE Design Agency. May participate in external working groups and/or assist in organizing meetings.

What You Need

Minimum Job Requirements:

• Linux Expertise - Linux knowledge and experiences, including usage and commands.

• Compilers - Experience with linux compilers such as GNU, Nvidia HPCSDK, Intel, or LLVM/Clang.

• Programming Skills - At least 2 years experience in programming in Python and BASH or C/C++. Experienced in basic software engineering principles.

• Strong interpersonal Skills - Including demonstrated ability to work within a team environment and with customers.

• Communication Skills - Outstanding written and oral technical communication. Ability to work with groups of people of various levels of technical knowledge or understanding.

• HPC Computing Experience - Knowledge of High Performance Computing, their environments and supporting infrastructure. Knowledge of distributed systems, including system architectures, computer networks, and software.

Additional Job Requirements for Scientist 3:

In addition to the job requirements outlined above qualification at the higher level requires:

• Programming Skills - Demonstrated/recorded advanced knowledge and experience in one or more of the programming languages in Python, Bash, and/or C/C++.

• Parallel Programming - Demonstrated/recorded experience programming in a parallel computing environment with MPI programming models, threads, or both

• Advanced Linux Experience - At least 5 years of experience working with manually compiled software in a Linux environment.

• HPC Computing Experience - At least 5 years working in a High Performance Computing environment. Knowledge of HPC networking and parallel file systems (Lustre, GPFS, etc.), experience with archive solutions (HPSS, TSM, etc.), experience with data movement tools.

Education/Experience at Scientist 2 level: Position requires a Bachelor's degree in a STEM field from an accredited college and university and 4 years of related experience, typically with experience at a university or National Lab or equivalent experience directly related to the occupation.

Education/Experience at Scientist 3 level: Position requires a Master's degree in a STEM field from an accredited college or university and 6 years of relevant experience or an equivalent combination of education and experience directly related to the occupation.

Desired Qualifications:

Advanced Linux System Administration

Linux system administration experience in a production environment

Linux Configuration Management experience (Ansible, Puppet, etc)

Knowledge of the intricacies of linking C and/or Fortran executables.

HPC Package Management Experience

Experience installing and configuring software with Spack or similar software

HPC Debugging

Experience with tools and methods for optimization and debugging in a highly parallel environment.

DevOps & Software Development

Experience with DevOps and continuous integration (CI) tools.

Visualization Software

Experience with visualization software and tools such as Ensight, Paraview, Visit

Linux Containers

Experience using container and container runtime software technologies such as Docker, Shifter, Charliecloud

Machine Learning and/or Artificial Intelligence

Experience ML/AI programming or toolkits such as PyTorch, Tensorflow, SciPy, NLTK

DOE/NNSA Applications

Experience with DOE/NNSA integrated codes.

Work Location:

This position will be located in Los Alamos, NM, with the potential for a hybrid work arrangement (60% onsite/40% offsite) from a location within 2 hours ground commute of this location. Reporting onsite will be required. Hybrid is at the discretion of management and can change at any time with appropriate notice.

Position commitment: Regular appointment employees are required to serve a period of continuous service in their current position in order to be eligible to apply for posted jobs throughout the Laboratory. If an employee has not served the time required, they may only apply for Laboratory jobs with the documented approval of their Division Leader. The position commitment for this position is 1 year.

Note to Applicants:

For consideration, applicants should submit a cover letter addressing how their knowledge, skills and abilities meet the minimum requirements along with a resume.

Where You Will Work

Located in beautiful northern New Mexico, Los Alamos National Laboratory (LANL) is a multidisciplinary research institution engaged in strategic science on behalf of national security. Our generous benefits package includes:

§ PPO or High Deductible medical insurance with the same large nationwide network

§ Dental and vision insurance

§ Free basic life and disability insurance

§ Paid childbirth and parental leave

§ Award-winning 401(k) (6% matching plus 3.5% annually)

§ Learning opportunities and tuition assistance

§ Flexible schedules and time off (PTO and holidays)

§ Onsite gyms and wellness programs

§ Extensive relocation packages (outside a 50 mile radius)

Additional Details

Directive 206.2 - Employment with Triad requires a favorable decision by NNSA indicating employee is suitable under NNSA Supplemental Directive 206.2 (https://directives.nnsa.doe.gov/supplemental-directive/sd-0206-0002) . Please note that this requirement applies only to citizens of the United States. Foreign nationals are subject to a similar requirement under DOE Order 142.3A.

Clearance: Q (Position will be cleared to this level). Selected applicants will be subject to a background investigation conducted by or on behalf of the Federal Government, and must meet eligibility requirements* for access to classified matter. This position requires a Q clearance. and obtaining such clearance requires US Citizenship except in extremely rare circumstances. Dependent upon the position, additional authorization to access classified information may be required, which may or may not be available to dual citizens. Receipt of a Q clearance and additional access authorization ultimately is a decision of the Federal Government and not of Triad.

*Eligibility requirements: To obtain a clearance, an individual must be at least 18 years of age; U.S. citizenship is required except in very limited circumstances. See DOE Order 472.2 (https://www.directives.doe.gov/directives-documents/400-series/0472.2-BOrder-chg1-pgchg) for additional information.

New-Employment Drug Test: The Laboratory requires successful applicants to complete a new-employment drug test and maintains a substance abuse policy that includes random drug testing. Although New Mexico and other states have legalized the use of marijuana, use and possession of marijuana remain illegal under federal law. A positive drug test for marijuana will result in termination of employment, even if the use was pre-offer.

Regular position: Term status Laboratory employees applying for regular-status positions are converted to regular status.

Internal Applicants: Regular appointment employees who have served the required period of continuous service in their current position are eligible to apply for posted jobs throughout the Laboratory. If an employee has not served the required period of continuous service, they may only apply for Laboratory jobs with the documented approval of their Division Leader. Please refer to Policy Policy P701 (https://int.lanl.gov/policy/documents/P701.pdf) for applicant eligibility requirements.

Equal Opportunity: Los Alamos National Laboratory is an equal opportunity employer and supports a diverse and inclusive workforce. All employment practices are based on qualification and merit, without regard to race, color, national origin, ancestry, religion, age, sex, gender identity, sexual orientation, marital status or spousal affiliation, physical or mental disability, medical conditions, pregnancy, status as a protected veteran, genetic information, or citizenship within the limits imposed by federal laws and regulations. The Laboratory is also committed to making our workplace accessible to individuals with disabilities and will provide reasonable accommodations, upon request, for individuals to participate in the application and hiring process. To request such an accommodation, please send an email to applyhelp@lanl.gov or call 1-505-664-6947 option 2 and then option 3.