Yes, you heard it right ! No more waiting for your laptop or desktop to finish training your favorite language model. You can now do all the heavy lifting directly from your phone!
Now before we dive into the details of how to set this up, let’s first talk about why anyone would want to do such a thing in the first place. Well, for starters, it’s pretty cool and impressive that you can run LLaMA on your phone! But more importantly, it allows you to have access to your language model wherever you go without having to carry around a laptop or desktop with you. This is especially useful if you’re traveling or working remotely from different locations.
So how do we get started? If it isn’t already, head over to the F-Droid website and download the app. Once you have Termux up and running, open it up and type in the following command:
#!/bin/bash # This line specifies the interpreter to be used for executing the script
# This script updates and upgrades the packages in Termux
pkg update # This command updates the list of available packages
pkg upgrade # This command upgrades the installed packages to their latest versions
This will ensure that all of your packages are up to date before we proceed with installing LLaMA. Next, let’s download the latest version of LLaMA from Hugging Face using wget:
# This line uses the wget command to download a file from a specified URL
wget https://huggingface.co/TheBlokeAI/llama-2-70B-chat/resolve/main/modelcard.md
# The -O flag is used to specify the output file name, in this case "modelcard.md"
# The -q flag is used to suppress any unnecessary output
# The -N flag is used to only download the file if it is newer than the existing one
# The -P flag is used to specify the directory where the file will be saved, in this case the current directory
wget -O modelcard.md -q -N -P . https://huggingface.co/TheBlokeAI/llama-2-70B-chat/resolve/main/modelcard.md
This will download the model card for LLaMA, which contains all of the important information about the language model such as its size and performance metrics. Once that’s done, let’s extract the contents of the file using unzip:
# This script downloads the model card for LLaMA and extracts its contents using unzip.
# Download the model card for LLaMA
wget https://example.com/modelcard.md
# Unzip the downloaded file
unzip modelcard.md
# The contents of the model card are now extracted and can be accessed for further use.
This will create a new directory called ‘llama-2-70B-chat’ in your current working directory with all of the necessary files for running LLaMA on Android. Now that we have everything set up, let’s run LLaMA using Python:
# This script is used to run LLaMA on Android using Python.
# The following line imports the necessary module for running LLaMA in C++.
python -m llama_cpp
# The --model flag specifies the location of the LLaMA model file.
--model llama-2-70B-chat/llama-2-70B-chat.bin
# The --tokenizer flag specifies the location of the LLaMA tokenizer file.
--tokenizer llama-2-70B-chat/llama-2-70B-chat.vocab
# The --input flag specifies the input file for LLaMA to generate responses from.
--input input.txt
# The --output flag specifies the output file where LLaMA's responses will be saved.
--output output.txt
This will run LLaMA using the ‘llama_cpp’ script, which is included in the downloaded files from Hugging Face. The ‘-m’ flag tells Python to execute a module (in this case, llama_cpp), and the remaining arguments are passed as options to that module. In our case, we’re specifying the path to the language model (‘llama-2-70B-chat/llama-2-70B-chat.bin’), tokenizer (‘llama-2-70B-chat/llama-2-70B-chat.vocab’), input file (‘input.txt’), and output file (‘output.txt’).
And that’s it! You should now have a fully functional LLaMA model running on your Android phone using Termux (F-Droid)! Of course, there are some limitations to this setup for example, the performance of LLaMA will be significantly slower than if you were running it on a laptop or desktop due to the limited resources available on most smartphones. But hey, at least now you can impress your friends with your newfound AI skills while waiting in line at the grocery store!