Senior Software Engineer

Senior Engineer, OpenShift Platform Performance and Scale

The Red Hat Performance and Scale Organization is seeking a seasoned senior engineer to join our OpenShift Platform Performance and Scale Team. This role involves testing, measuring, and analyzing the performance and scalability of Red Hat OpenShift, the leading application modernization platform built on Kubernetes, to support the onboarding and management of AI workloads on OpenShift.

OpenShift serves as the foundation for OpenShift AI, the platform designed for model training, tuning, serving, inferencing, development, and MLOps in a hybrid cloud environment. As such, it plays a vital role in Red Hat's AI strategy and offerings. This role will focus on performance and scale testing of features and solutions geared towards emerging workloads like AI and Data/Analytics to ensure that Red Hat OpenShift can meet the demands of modern applications. The engineer will leverage their knowledge of systems, AI and hardware accelerator performance to theorize bottlenecks and limitations, devise test plans, execute workloads, collect and analyze data, and communicate findings. This role will require the ability to work cross-functionally with product management, engineering leadership, development teams, and quality engineers to measure performance, clearly articulate findings, and address bottlenecks. Time will also be spent collaborating with software engineering teams on bug fixes, code optimization, and resource usage reduction, as well as developing open-source tools for the reliability and repeatability of tests. This is a unique opportunity to work at the intersection of cutting-edge hardware and software!

The broader mission of the Performance and Scale organization is to establish performance and scale leadership across the Red Hat product and cloud services portfolio. The scope includes component-level, system, and solution analysis and targeted enhancements. The team collaborates with engineering, product management, product marketing, customer support, and Red Hat's hardware and software ecosystem partners.

At Red Hat, our commitment to open source innovation extends beyond our products - it's embedded in how we work and grow. Red Hatters embrace change especially in our fast-moving technological landscape and have a strong growth mindset. That's why we encourage our teams to proactively, thoughtfully, and ethically use AI to simplify their workflows, cut complexity, and boost efficiency. This empowers our associates to focus on higher-impact work, creating smart, more innovative solutions that solve our customers' most pressing challenges.

What You Will Do

• Work closely with management, product owners, developers, and quality engineers to understand product requirements and build suitable test plans to verify the performance and scale of OpenShift features and solutions for running AI workloads, such as Kubernetes Dynamic Resource Allocation (DRA), autoscaling, and operators for detection, configuration, and management of AI accelerators.
• Develop sophisticated tests that simulate user workloads through comprehensive end-to-end automation, leveraging custom-built and state-of-the-art open-source tools and frameworks.
• Deep dive into performance issues with the intent of discovering their root causes in complex distributed systems.
• Design and develop monitoring and reporting tools for performance and scale tests and analysis.
• Document your research and results clearly and concisely, and communicate findings both internally and externally.
• Engage in upstream communities to help test performance and scale early and influence design and development decisions.
• Triage, debug, and root cause customer issues related to OpenShift performance and scale.
• Present your work and findings at internal and external conferences.
• Collaborate with cross-functional teams to identify opportunities for AI integration within the software development lifecycle, driving continuous improvement and innovation in engineering practices; share use cases for successful experiments with stakeholders for broader use.

What You Will Bring

• Master's Degree in Computer Science or a related field with 3+ years of relevant experience, or a Bachelor's Degree in Computer Science or a related field with 5+ years of relevant experience.
• Demonstrable experience, understanding, and passion for performance engineering.
• Working knowledge of Kubernetes or OpenShift.
• Strong programming, debugging, and profiling skills in Python and/or Golang.
• Hands-on experience with performance measurement, analysis, and optimization.
• Experience with distributed systems.
• Very strong Linux system administration and system engineering skills.
• Solid scripting skills, particularly with Bash, Python, or Ansible.
• Experience working with public clouds like AWS, Azure, GCP, or IBM Cloud, as well as bare metal environments.
• Experience analyzing and interpreting large volumes of test results and succinctly communicating findings through easy-to-understand graphs/charts.
• Experience with collaborative software development methodologies, tools, and version control.
• Knowledge of statistical analysis and experimental design techniques.
• Excellent communication and interpersonal skills.
• Ability to work independently and proactively seek collaboration.

The Following Are Considered a Plus:

• Experience with container technologies like Podman or Docker, and familiarity with building container images.
• Experience with system performance engineering and metrics collection tools like iostat, vmstat, sar, perf, and Prometheus.
• Experience with monitoring and dashboarding tools like Prometheus and Grafana.
• Experience with AI accelerators and tools for monitoring/managing their usage.
• A demonstrated history of contributing to open-source projects.
• Presentation skills and public speaking abilities for conferences and demonstrations.

The salary range for this position is $111,260.00 - $183,580.00. Actual offer will be based on your qualifications.

Red Hat determines compensation based on several factors including but not limited to job location, experience, applicable skills and training, external market value, and internal pay equity. Annual salary is one component of Red Hat's compensation package. This position may also be eligible for bonus, commission, and/or equity. For positions with Remote-US locations, the actual salary range for the position may differ based on location but will be commensurate with job duties and relevant work experience.

Red Hat is the world's leading provider of enterprise open source software solutions, using a community-powered approach to deliver high-performing Linux, cloud, container, and Kubernetes technologies. Spread across 40+ countries, our associates work flexibly across work environments, from in-office, to office-flex, to fully remote, depending on the requirements of their role. Red Hatters are encouraged to bring their best ideas, no matter their title or tenure. We're a leader in open source because of our open and inclusive environment. We hire creative, passionate people ready to contribute their ideas, help solve complex problems, and make an impact.

Comprehensive medical, dental, and vision coverage Flexible Spending Account - healthcare and dependent care Health Savings Account - high deductible medical plan Retirement 401(k) with employer match Paid time off and holidays Paid parental leave plans for all new parents Leave benefits including disability, paid family medical leave, and paid military leave Additional benefits including employee stock purchase plan, family planning reimbursement, tuition reimbursement, transportation expense account, employee assistance program, and more!

These benefits are only applicable to full time, permanent associates at Red Hat located in the United States.

Red Hat's culture is built on the open source principles of transparency, collaboration, and inclusion, where the best ideas can come from anywhere and anyone. When this is realized, it empowers people from different backgrounds, perspectives, and experiences to come together to share ideas, challenge the status quo, and drive innovation. Our aspiration is that everyone experiences this culture with equal opportunity and access, and that all voices are not only heard but also celebrated. We hope you will join our celebration, and we welcome and encourage applicants from all the beautiful dimensions that compose our global village.

Red Hat is proud to be an equal opportunity workplace and an affirmative action employer. We review applications for employment without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, ancestry, citizenship, age, veteran status, genetic information, physical or mental disability, medical condition, marital status, or any other basis prohibited by law.