Nomic Embed Code: A State-of-the-Art Code Retriever

Nomic Embed Code: A State-of-the-Art Code Retriever

NEWSMarch 27, 2025

With the rise in AI code assistants like Cursor and Windsurf, effective code retrieval is critical for improving code generation across large codebases. Existing code embedding models tend to struggle with real-world challenges like finding bugs in Github repositories, likely due to noisy inconsistent data used during training.

As an extension to our ICLR 2025 paper CoRNStack: High-Quality Contrastive Data for Better Code Retrieval and Reranking, we've scaled up and trained Nomic Embed Code, a 7B parameter code embedding model which outperforms Voyage Code 3 and OpenAI Embed 3 Large on CodeSearchNet. Like all of Nomic's models, Nomic Embed Code is truly open source. The training data, code, and model weights are all available on Huggingface and Github under an Apache-2.0 license.

The CoRNStack

CoRNStack, is a large-scale high-quality training dataset specifically curated for code retrieval. The dataset is filtered using dual-consistency filtering and utilizes a novel sampling technique to progressively introduce harder negative examples during training.

In CoRNStack, we trained and released Nomic CodeRankEmbed, a 137M parameter code embedding model, and Nomic CodeRankLLM, a 7B parameter code reranking model. Now, we've unified the retrieval and reranking models into a streamlined 7B parameter model, Nomic Embed Code.

Nomic Embed Code

Nomic Embed Code is a 7B parameter code embedding model that achieves state-of-the-art performance on CodeSearchNet.

__wf_reserved_inherit
Type image caption here (optional)

Like our previous models, Nomic Embed Code is truly open source. The training data, code, and model weights are all available on Huggingface and Github.

Curating the CoRNStack

Nomic Embed Code's impressive performance is due to the quality of the training data. Starting with the deduplicated Stackv2, we create text-code pairs from function docstrings and respective code. We filtered out low-quality pairs where the docstring wasn't English, was too short, or that contained URLs, HTML tags, or invalid characters. We additionally kept docstrings with text lengths of 256 tokens or longer to help the model learn long-range dependencies.

After the initial filtering, we used dual-consistency filtering to remove potentially noisy examples. We embed each docstring and code pair and compute the similarity between each docstring and every code example using an out of the box embedding model. We remove pairs from the dataset if the corresponding code example is not found in the top-2 most similar examples for a given docstring.

__wf_reserved_inherit

During training, we employ a novel curriculum-based hard negative mining strategy to ensure the model learns from challenging examples. We use a softmax-based sampling strategy to progressively sample hard negatives with increasing difficulty over time.

Conclusion

In this blog post, we've introduced Nomic Embed Code, a 7B parameter code embedding model that achieves state-of-the-art performance on CodeSearchNet. Nomic Embed Code builds on the CoRNStack dataset and training methodology, which are detailed in our ICLR 2025 paper. Code embeddings represent just one part of the Nomic Embed Ecosystem. To learn more about the entire ecosystem, check out our blog.

Share this article:

Related Articles

Ask for it, and Nomic does it

The drawings, DWGs, schedules, and reviews Nomic does from your project files.

NomicPower layout
E-301 Power Plan Level 3.dwgDrawn by Nomic

AI CAD Drafting

Ask for the plan and get DWGs in your layers and text styles.

NomicDrawing review
Ashburn DC-3 100% CD Set.pdf2 findings

Automated Drawing Review

Coordination conflicts flagged on the sheet before issue.

NomicSchematic drawing set
M-101 HVAC Plan.dwgDrawn by Nomic

Schematic Drawing Sets

Architectural background in. Floor, plumbing, and HVAC sheets out.

NomicSubmittal review
08 44 13-01 Curtain Wall Shop Drawings.pdf3 findings

Automated Submittal Review

Shop drawings checked against the spec, cited to the page.

NomicReview comment response
M-501 Mechanical Details.dwgDrawn by Nomic

Review Comment Responses

Review letter in. A response and revised DWGs out.

NomicBIM takeoff
LEVEL 1LEVEL 2LEVEL 3LEVEL 4ROOF
HPMO-Structure.ifc

AI for BIM & IFC Models

Ask an IFC model for quantities and missing data.

NomicEnergy code schedules
Space and Lighting Schedules.xlsxDrawn by Nomic

Energy Code Schedules

Model spaces and lighting in. An energy-code import file out.

NomicCode compliance
A-201 Second Floor Plan.pdf1 finding

Code Compliance Review of Drawings

Drawings checked against 380+ codes, with citations.

NomicSurvey to DWG
M-001 Existing Conditions.dwgDrawn by Nomic

Existing Conditions Surveys

Site walk photos and notes in, a to-scale DWG and report out.

NomicDetail search
Lakeview Library CD Set.pdf

Firm-Wide Detail Search

Find the detail your firm already drew.

NomicRevit-ready DWG
P-201 Domestic Water Reroute.dwgDrawn by Nomic

Revit-Ready Deliverables

Coordination comments drawn as a DWG to link into your model.

NomicRFI response
G-003 Life Safety Plan.pdf

Project Research

Answers from every drawing, spec, and email on the job.

Nomic agents work in your project delivery software and tools
SharePoint
Egnyte
Autodesk Forma
Revit
AutoCAD
Procore
ProjectWise (Bentley)
Trimble ProjectSight
Fieldwire
Google Drive
Box
Dropbox
Microsoft Teams
Outlook
Gmail
Nomic Embed Code: A State-of-the-Art Code Retriever | Nomic News