Using Starcoder for code completion in VS Code using HF’s MLCommons Model Hub

in

Before anything else, what exactly is going on here. Essentially, Starcoder is a pre-trained language model that can help you write code faster and more accurately by suggesting possible completions for your incomplete lines of code. And HF’s MLCommons Model Hub is where we go to find these models and use them in our projects.

So how do we get started? Well, first we need to download the Refact plugin from the VS Code marketplace (which you can do by clicking on “Extensions” in the left-hand menu, searching for “Refact”, and then installing it). Once that’s done, we can open a new file in our project and start writing some code.

But wait! Before we get too excited, how to configure this bad boy so that Starcoder knows where to find its model (which is stored on HF’s MLCommons Model Hub). To do this, we need to run a local server using the following command: 

# This line runs the python module "mlc_chat.rest" to start a local server.
python -m mlc_chat.rest \
# This flag specifies the model to be used by the server.
--model dist/starcoder1B-v2-personal-copilot-merged-q4f16_1/params \
# This flag specifies the path to the library containing the model.
--lib-path dist/starcoder1B-v2-personal-copilot-merged-q4f16_1/starcoder1B-v2-personal-copilot-merged-q4f16_1-metal.so

This will start a local server that listens for requests from our VS Code extension, which we’ll configure next.

To do this, open the “Settings” menu in VS Code (which you can access by clicking on the gear icon in the left-hand toolbar), and then search for “Inference Endpoints”. Once you find it, click on “Add Configuration”, and then enter the following information: 

{
    "endpoint": "https://example.com/inference", // replace this with the URL of your inference endpoint
    "modelName": "starcoder1B-v2-personal-copilot-merged-q4f16_1", // replace this with the name of your model (which you can find on HF's MLCommons Model Hub)
}

// This is a json script used to configure the inference endpoint and model name for the VS Code extension.

{
    "endpoint": "https://example.com/inference", // replace this with the URL of your inference endpoint
    "modelName": "starcoder1B-v2-personal-copilot-merged-q4f16_1", // replace this with the name of your model (which you can find on HF's MLCommons Model Hub)
    "settings": {
        "inferenceEndpoints": [ // list of inference endpoints
            {
                "name": "My Endpoint", // name of the endpoint
                "url": "https://example.com/inference", // URL of the endpoint
                "modelName": "starcoder1B-v2-personal-copilot-merged-q4f16_1" // name of the model used for this endpoint
            }
        ]
    }
}

// This is a json script used to configure multiple inference endpoints and their corresponding model names for the VS Code extension.

Once that’s done, save and close the settings file.

Now we’re ready to start using Starcoder for code completion! Just open a new file in VS Code, paste some code (or just type out an incomplete line), and then hit “Tab” or “Enter”. The magic happens, and you should see suggestions popping up below your cursor.

Starcoder is not only great for code completion, but it can also help you write better comments and documentation by suggesting possible completions based on the context of your code. And if that wasn’t enough, it can even act as a chatbot (sorta) by answering questions about your code or providing explanations when needed.

Using Starcoder for code completion in VS Code using HF’s MLCommons Model Hub. It may sound complicated at first, but trust us once you get the hang of it, you won’t be able to live without it!

SICORPS