Note: The job is a remote job and is open to candidates in USA. STAND 8 Technology Consulting provides end-to-end IT solutions to enterprise partners across the United States. They are looking for an AI/ML Prompt Scientist/Engineer to design, evaluate, and optimize prompt-based systems that drive real-world impact across cutting-edge AI applications.
Responsibilities
• Design, optimize, and evaluate prompts and orchestration pipelines to enhance LLM performance and scalability, supporting both Data Scientist and engineering workflows.
• Implement programmatic prompting using frameworks such as LangChain, LlamaIndex, DSPy, Guidance, and LangGraph.
• Define and manage evaluation frameworks, including A/B testing, human and automated scoring, and model quality assessments using tools such as Ragas, UpTrain, DeepEval, TruLens, Promptfoo, and OpenAI Evals.
• Engineer and maintain context management systems, including conversation memory, retrieval logic, and context compression strategies (chunking, summarization, prioritization).
• Design and optimize retrieval and RAG pipelines, integrating vector databases (e.g., Weaviate) and rerankers (e.g., Cohere) to enhance relevance and accuracy.
• Establish observability and traceability mechanisms (e.g., Langfuse) to monitor performance metrics, latency, and operational costs.
• Implement guardrails, safety mechanisms, and bias mitigation policies using frameworks such as Guardrails AI, NeMo Guardrails, Outlines, and Rebuff.
• Collaborate with cross-functional engineering and research teams-including Data Scientists, ML, and AI engineering groups-to ensure high-quality, compliant, and well-documented system delivery.
Skills
• Proven experience in LLM and NLP model operations, including tokenization, context windows, and safety considerations.
• Strong technical expertise in prompt engineering, orchestration frameworks, and context management.
• Hands-on experience with retrieval-augmented generation, evaluation frameworks, and observability tooling.
• Understanding of key evaluation metrics such as faithfulness, groundedness, coherence, toxicity, latency, and cost.
• Experience with compliance and ethical AI practices, including bias and hallucination mitigation.
• Excellent analytical, documentation, and communication skills, with the ability to collaborate effectively across teams.
• Model fine-tuning and alignment (e.g., LoRA, DPO/RLHF, preference tuning).
• Experience developing AI agents and integrating tool-use capabilities.
Benefits
• Medical coverage and Health Savings Account (HSA) through Anthem
• Dental/Vision/Various Ancillary coverages through Unum
• 401(k) retirement savings plan
• Paid-time-off options
• Company-paid Employee Assistance Program (EAP)
• Discount programs through ADP WorkforceNow
Company Overview
• STAND 8 is a global technology consulting and staffing leader that helps businesses transform and scale through tailored IT solutions. It was founded in 2009, and is headquartered in Long Beach, California, USA, with a workforce of 201-500 employees. Its website is https://www.stand8.io.
Company H1B Sponsorship
• STAND 8 Technology Consulting has a track record of offering H1B sponsorships, with 1 in 2024, 5 in 2023, 3 in 2022, 6 in 2020. Please note that this does not guarantee sponsorship for this specific role.
