The simplest way to run LLaMA or Alpaca on your local machine with no Code (Mac/Linux)

LLaMA and Alpaca are language models that behave similarly to OpenAI's ChatGPT, but are available to the public for free. They behave slightly differently. LLaMA was released by Meta, and Alpaca is an optimized version of LLaMA trained by leveraging OpenAI's ChatGPT. We prefer using LLaMA as it is optimized and slightly more user friendly when dealing with prompts. Here's how to run either LLaMA or Alpaca on any computer.

Don't worry if you're not a technical expert. Our guide is easy to follow, and we provide step-by-step instructions for each stage of the process. By the end, you'll have a fully functional Alpaca model running, ready for natural language processing tasks like chatbots and language translation.

Hardware Requirements

Disk Space

Alpaca

There are 2 different models available: 7B and 13B

7B

Alpaca comes fully quantized (compressed), and the only space you need for the 7B model is 4.21GB

13B

Alpaca comes fully quantized (compressed), and the only space you need for the 13B model is 8.14GB

LLaMA

The models are fairly large files. The model name must be one of: 7B, 13B, 30B, and 65B.

You do NOT have to install all models, choose the one appropriate given your disk space. Larger models tend to have better results, however the 7B model works just fine. Let's take a look at how much space each model takes up:

7B

• Full: The model takes up 31.17GB
• Quantized: 4.21GB

13B

• Full: The model takes up 60.21GB
• Quantized: 4.07GB * 2 = 8.14GB

30B

• Full: The model takes up 150.48GB
• Quantized: 5.09GB * 4 = 20.36GB

65B

• Full: The model takes up 432.64GB
• Quantized: 5.11GB * 8 = 40.88GB

Installation Instructions

If you on either a mac or linux, here's a link to the installation instructions for Windows. View post here

Mac

Step 1. Install node.js >= 18

If your mac doesn't have node.js installed yet, make sure to install node.js >= 18

• Install nvm, node version manager, which helps to make managing node installation and versioning painless
• Install node

Step 2. Install models

Currently supported engines are

Copy to clipboard and

...

Copy to clipboard.

Add alpaca models

To download alpaca models, you can run:

Add llama models

To download alpaca models, you can run:

or to download multiple models:

Step 3. Run Web UI

After everything has been installed, run the following command to launch the web UI server:

now open

Copy to clipboard in your browser. Enjoy!

Here's what it should look like when it's working: