by In this tutorial, we explain how to install and run AI models locally by using LocalAI. One of the missions of this website and our YouTube channel is to teach how to run open-source AI models locally and how to build open-source machine learning and AI applications. So far, we have extensively covered the Ollama framework for running models locally. However, it is always a good idea to diversify the strategies for testing and running AI models locally.
One of the important alternatives for running models locally is LocalAI. In this tutorial, we explain how to install LocalAI and how to run AI models locally by using LocalAI. LocalAI is the free, Open Source OpenAI alternative. By using LocalAI you can locally run LLMs, generate images, generate audio, and other AI tasks on consumer-grade hardware. You can even run models on CPUs. However, we suggest to install a GPU on your system. The YouTube tutorial is given below.
Installation Instructions
You need to have Linux Ubuntu installed on your system. Open a Linux terminal and verify your Linux version
lsb_release -aThe output should be Linux Ubuntu.
The first step is to install Curl. To do that type this
sudo apt update && sudo apt upgrade
sudo apt install curl
curl --versionNext, check if the CUDA Toolkit is installed. To do that, type
nvcc --version
If you get a response, then the CUDA Toolkit and CUDA compilers are installed. If not, follow the tutorial given on this webpage to install the CUDA Toolkit.
The next step is to install Docker Engine. To do that, follow the instructions on this webpage:
https://docs.docker.com/engine/install/ubuntu
Once Docker is installed, install LocalAI by using the Docker approach:
docker run -ti --name local-ai -p 8080:8080 localai/localai:latestThis will install LocalAI by using Docker, and it will download several models. Next, click on the link shown in the terminal after LocalAI is installed, and LocalAI will start in a web browser. For more details see the YouTube tutorial.
