Who You Are (Required):
- Bachelor’s Degree in Information Technology, Computer Science, Computer Engineering, Electrical Engineering, Systems Engineering, Physics, Math, or equivalent full-time professional experience; Master’s Degree in Engineering or other technical field highly desired
- 5-10+ years of professional experience in software engineering, machine learning engineering, AI engineering, or related technical roles
- Proficiency in Python, including experience with core libraries such as NumPy and Pandas
- Deep hands-on experience with production local inference engines such as vLLM, SGLang, Triton Inference Server, or TensorRT-LLM
- Experience building software with one or more modern AI/ML frameworks such as PyTorch, TensorFlow, LangChain, LangGraph, Semantic Kernel, or AutoGen
- Experience with Linux and hardening (fapolicy/selinux/fips/etc)
- Experience with commercially available AI tooling/chat
- Experience with hosting LLM servers
- Experience with hosting different models (chat/embedding)
- Experience with distributed networking (reverse proxy/load balancing/firewalls/etc)
- Experience with containerization
- Ability to work independently on technical tasks while collaborating effectively in a team environment
- Ability to shift from one project to another in an agile work environment
- Strong leadership capabilities and skills
- Strong documentation skills
- Ability to travel up to 10% of the time, as needed
What Makes You Stand Out:
- Experience hosting LLM servers on local hardware
- Experience with various GPU architectures (NVIDIA preferred)
- Experience with AI Gateways
- Experience with LLM servers and model optimization (VRAM/layers/quantization/fasttensor/kv-cache/parallel/etc)
- Experience with LLM toolcalling
- Experience with RAG
- Experience with vector stores
- Experience with ModelContextProtocol
- Experience with container orchestration
- Experience managing and directing personnel while maintaining cost and schedule targets
- Problem-solving skills with the ability to navigate ambiguous situations
If you see yourself reflected in this role and are excited about the impact you could make, we encourage you to apply! If you know someone who may be a great fit, please feel free to share this opportunity with your network.
Pay Range*: $114,000 - $184,000 or $135,000 - $231,000
This amount may not be reflective of actual compensation that may be earned as pay is dependent on a candidate’s experience, skills, and education. The posted range does not include bonuses, commissions, tips, or other benefits. IDT is often looking to place multiple candidates at various levels. Therefore, more than one pay range has been included, commensurate with experience.
