Research Scientist Intern, Real-Time Multimodal AI (PhD)

Reality Labs is building the future of connection through world-class AR/VR hardware and software. The XR Tech AIX (AI Experiences) team is developing cutting-edge real-time AI systems that power next-generation communication experiences. We are creating intelligent agents that seamlessly interface with fine-tuned foundation models to enable rich, real-time interactions in video calling and telepresence scenarios.We are seeking an exceptional Research Scientist Intern to join our team and contribute to the development of real-time multimodal AI systems. This role focuses on fine-tuning and optimizing large foundation models-particularly vision-language models-for real-time agent-based applications. You will work at the intersection of multimodal learning, real-time systems, and agentic AI.Our internships are twelve (12) to twenty-four (24) weeks long with a flexible summer start date. Research Scientist Intern, Real-Time Multimodal AI (PhD) Responsibilities: Research and develop novel approaches for fine-tuning large multimodal foundation models (vision-language, audio-visual) for real-time applicationsDesign and implement efficient inference pipelines for deploying fine-tuned models in real-time communication scenariosExplore agentic architectures that leverage fine-tuned models as tools within larger AI systemsCollaborate with cross-functional teams to integrate models into prototype experiencesDocument and present research progress with the goal of publishing findings at top-tier ML/CV conferencesContribute to building working prototypes that demonstrate the capabilities of fine-tuned multimodal modelsMinimum Qualifications:2+ years of research experience in one or more of the following areas: multimodal learning, vision-language models, large language models, or foundation model fine-tuningHands-on experience fine-tuning large foundation models (e.g., LLaVA, InternVL, Qwen-VL, LLaMA, or similar)Strong programming skills in PythonExperience with deep learning frameworks such as PyTorchExcellent communication skills and ability to work independentlyMust obtain work authorization in the country of employment at the time of hire, and maintain ongoing work authorization during employmentPreferred Qualifications:Proven track record of achieving significant results as demonstrated by first-authored publications at leading conferences such as NeurIPS, ICML, ICLR, CVPR, ICCV, ECCV, ICASSP, Interspeech, ACL, EMNLP, or similarExperience with speech-to-speech LLMs or audio-visual foundation modelsFamiliarity with real-time communication systems (e.g., LiveKit, WebRTC) or low-latency inference optimizationExperience with cloud infrastructure (AWS) and containerization (Docker)Experience with parameter-efficient fine-tuning techniques (LoRA, QLoRA, adapters, etc.)Experience with agentic AI systems, tool-use, or function-calling in LLMsDemonstrated software engineering experience via internships, work experience, or contributions to open source repositories (e.g., GitHub)Intent to return to degree program after completion of the internshipAbout Meta: Meta builds technologies that help people connect, find communities, and grow businesses. When Facebook launched in 2004, it changed the way people connect. Apps like Messenger, Instagram and WhatsApp further empowered billions around the world. Now, Meta is moving beyond 2D screens toward immersive experiences like augmented and virtual reality to help build the next evolution in social technology. People who choose to build their careers by building with us at Meta help shape a future that will take us beyond what digital connection makes possible today-beyond the constraints of screens, the limits of distance, and even the rules of physics.Meta is proud to be an Equal Employment Opportunity and Affirmative Action employer. We do not discriminate based upon race, religion, color, national origin, sex (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender, gender identity, gender expression, transgender status, sexual stereotypes, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics. We also consider qualified applicants with criminal histories, consistent with applicable federal, state and local law. Meta participates in the E-Verify program in certain locations, as required by law. Please note that Meta may leverage artificial intelligence and machine learning technologies in connection with applications for employment. Meta is committed to providing reasonable accommodations for candidates with disabilities in our recruiting process. If you need any assistance or accommodations due to a disability, please let us know at [email protected].$7,650/month to $12,134/month + benefitsIndividual compensation is determined by skills, qualifications, experience, and location. Compensation details listed in this posting reflect the base hourly rate, monthly rate, or annual salary only, and do not include bonus, equity or sales incentives, if applicable. In addition to base compensation, Meta offers benefits. Learn more about benefits at Meta.