Clinical AI Safety Evaluator - Physician / Clinician Part-Time | 5 Hours/Week | Remote | Flexible Schedule About the Role Help shape the future of healthcare AI. Highspring is seeking practicing clinicians to evaluate the accuracy, safety, and quality of generative AI healthcare applications for a Fortune 100 client. You'll use your clinical expertise to evaluate AI-generated responses, identify potential safety risks, and red-team AI models to uncover vulnerabilities before they reach patients and the public. This is a remote, part-time vendor opportunity with approximately 5 hours per week. What You'll Do • Evaluate AI-generated healthcare responses for clinical accuracy, safety, and quality. • Conduct red-team testing by creating realistic clinical scenarios and challenging AI models. • Identify hallucinations, unsafe recommendations, bias, and other potential clinical risks. • Provide clear, evidence-based rationale for evaluation scores and identified issues. • Apply structured evaluation rubrics consistently across a variety of healthcare scenarios. • Test AI responses from different patient perspectives, including complex or high-risk situations. What We're Looking For • MD, MBBS, or equivalent medical degree from an accredited institution. • Board Certification in Psychiatry, Clinical Psychology, Family Medicine, Primary Care, or Internal Medicine. • Active, unrestricted U.S. clinical license in good standing. • Current, hands-on patient care experience. • Experience using generative AI tools such as ChatGPT, Gemini, Claude, or similar platforms. • Strong analytical, clinical reasoning, and written communication skills. • Understanding of LLM limitations, including hallucinations, bias, sycophancy, and unsafe clinical recommendations. • Ability to create structured prompts and evaluate AI responses objectively. Preferred Qualifications • Experience with healthcare AI, AI safety, red teaming, LLM evaluation, Trust & Safety, RLHF, or medical informatics. • Advanced degree in a related field such as MPH, Medical Informatics, Computer Science, or Behavioral Science. • Multilingual skills and understanding of diverse cultural approaches to healthcare. Schedule • Remote • Approximately 5 hours per week • Monday-Friday, flexible hours • Daily check-in required • Remote until further notice About the Opportunity This is a unique opportunity to combine your clinical expertise with emerging AI technology and help improve the safety, accuracy, and reliability of healthcare AI. Highspring is an Equal Opportunity Employer and does not discriminate against any employee or applicant for employment because of race, color, sex, age, national origin, religion, sexual orientation, gender identity, status as a veteran, disability or any other federal, state or local protected class. By submitting to this position, you are agreeing to be included in our talent pool for future hiring for similarly qualified positions. Desired Skills and Experience - Active U.S. clinical license + MD, MBBS, or equivalent - Board Certification in Psychiatry, Clinical Psychology, Family Medicine, Primary Care, or Internal Medicine - Current hands-on patient care experience - Generative AI / LLM experience, including ChatGPT, Gemini, or Claude - AI safety, clinical evaluation, red teaming, and strong analytical/written communication skills Equal Opportunity Notice Highspring LLC (d/b/a Vaco by Highspring) and its parents, affiliates, and subsidiaries ("we," "our," or "Vaco by Highspring") are committed to the full inclusion of all qualified individuals and does not discriminate against any employee or applicant for employment because of race (including but not limited to traits historically associated with race such as hair texture and hair style), color, sex (includes pregnancy or related conditions), religion or creed, national origin, citizenship, age, disability, status as a veteran, union membership, ethnicity, gender, gender identity, gender expression, sexual orientation, marital status, political affiliation, or any other protected characteristics as required by applicable law. The company is also committed to ensuring that persons who need them are provided with reasonable accommodations; if an accommodation is needed to participate in the job application or interview process, please contact HR@vaco.com . Vaco by Highspring also wants all applicants to know their rights that workplace discrimination is illegal. Representation Notice By submitting to this position, you agree that you will be giving Vaco by Highspring the exclusive right to present your as a candidate for the foregoing employment opportunity. Additionally, you agree to be included in our talent pool for future hiring for similarly qualified positions. You further agree that you have represented information about yourself accurately and have not affirmatively misrepresented your qualifications. Lastly, you agree to maintain as confidential, to the fullest extent permitted by law, any information you learn from Vaco by Highspring about the position and you will limit disclosure of information about the position only to the extent necessary to perform any obligations in furtherance of your application. In exchange, Vaco by Highspring agrees to exercise reasonable efforts to represent you through all solicitation, job screening and resume dispersal. For residents of Ontario, Canada only: to the extent the position for employment is not with Highspring or not otherwise noted as vacant above, candidate should be informed that this role is to replace a presently employed person at Vaco by Highspring's client. Privacy Notice Vaco by Highspring respects your privacy and are committed to providing transparent notice of our policies. • California residents may access Vaco by Highspring HR Notice at Collection for California Applicants and Employees here. • Virginia residents may access our state specific policies here. • Residents of all other states may access our policies here. • Canadian residents may access our policies in English here and in French here. • Residents of countries governed by GDPR and UK GDPR may access our policies here. Additionally, submissions to this position are subject to the use of AI to perform preliminary candidate screenings, focused on ensuring minimum job requirements noted in the position are satisfied. More details about Vaco by Highspring's use of AI can be found here (https://www.highspring.com/ai-use-notices/). Further assessment of candidates beyond this initial phase will be conducted by recruiters and hiring managers. Vaco by Highspring does not know and cannot opine on if its client's use of AI products in hiring. Pay Transparency Notice Determining compensation for this role (and others) at Vaco by Highspring depends upon a wide array of factors including but not limited to: · the individual's skill sets, experience and training; · licensure and certification requirements; · office location and other geographic considerations; and · other business and organizational needs. With that said, as required by local law, Vaco by Highspring believes that the following salary range referenced above reasonably estimates the base compensation for an individual hired into this position in geographies that require salary range disclosure. The individual may also be eligible for discretionary bonuses and/or participation in medical, dental, and vision benefits as well as the company's retirement plan (or similar retirement benefits).