Scope: Set up and configure a local AI server for a civil engineering firm on a dedicated Windows workstation (92GB GPU VRAM). The stack includes WSL2, Docker, vLLM serving a 70B-parameter model (Qwen2.5-72B or Llama 3.3-70B), and AnythingLLM as a RAG document assistant. Internal network deployment for team access also required.
Deliverables:
- vLLM running reliably via Docker Compose with auto-restart
- AnythingLLM connected to local model and indexed with firm documents
- Internal network access configured for team
- Basic documentation of the setup for internal IT reference
Skills required: Linux/WSL2, Docker, NVIDIA CUDA, vLLM or similar LLM serving, RAG pipelines
Estimated effort: 2-5 days. Open to hourly or fixed-price proposals.
Note: All work involves sensitive engineering project data — candidate must be comfortable working within a fully on-premises, no-cloud-data-transfer constraint. This position is not eligible for remote work. The candidate must be local to Mason, OH.
Work Location: In person
