Introduction
In this blog, we delve into the how we harness the power of Large language models to build Translation App, highlighting its features, underlying technology, and the role of key libraries like Transformers and Django. The app is built using a pre-trained LLM model called flan-t5-base, which was released by Google. This model is pre-trained on a large text dataset without any filtration, making it highly versatile and also it was trained on multiple different languages make it ideal model to be used for this translation app.
Prompt Engineering
The secret to unlocking the true potential of LLMs lies in prompt engineering. Crafting precise and informational prompts influences the quality of model-generated outputs. In the context of our Translation App, prompt engineering plays a pivotal role in tailoring user inputs to elicit accurate translations. By fine-tuning prompts, we guide the model to comprehend user intent and deliver translations that resonate fluently with native speakers and accurate as possible.
We have to keep in mind all the LLM's are trained in a unique way and the prompts given during the training influences the model output in a great way so while choosing you own open source LLM's if the information regarding the prompts are not available it is best to avoid as creating a good prompt will be time consuming. In our case since the model is popular lot of research was done by many user to find how to create good prompts for this model. This helped us a lot by saving time by creating a good prompt for our app with only minimal trail and error.
Building the App
For a robust web application the Django framework was chosen as this choice empowers us to deliver a seamless and intuitive user experience. The frontend is meticulously designed, featuring a responsive layout that adapts effortlessly to diverse devices. Users are greeted with a user-friendly interface, complete with a dedicated text-area for input and an input field for language selection. Choosing Django had also a potential great advantage as it can be integrated with out other apps such as Chatbot etc... which were also build using Django, this enables us to combine such application and deploy them if needed.
Loading the Model
To use the model there are two part to it, first we have to use the Tokenizer to tokenize the words and ideally it should be similar to the one it has been trained on. Luckily for us Google has also provided the Tokenizer for the model which can be loaded using T5 Tokenizer class from the Transformers library of hugging face. Now in order to load the model we are using T5ForConditionalGeneration classes from the Transformers library. These components synergize to preprocess input text and generate accurate translations.
The T5 Tokenizer's role in structuring input and the T5ForConditionalGeneration's ability in producing coherent translations forms the backbone of our application. Before Tokenizing the user input we have to remember get the user input and required language the text is to be translated and place it within out prompt and then tokenize the whole prompt.
To get the output from the model we can use the generate function and give it the tokenized input and also specify some of the model parameters such max length of the output. Since this model is a relatively small model compared to the once such as GPT3, 3.5, Llama 2 etc... the ability of the model to generate very long text is slightly poor compared to above mentioned model so we are limiting ourselves to 2048 output tokens.
Here are some of the screenshots of the App.
For inquiries regarding the creation of similar applications utilizing open-source Large Language Models or the development of prototypes for various applications that leverage the capabilities of Large Language Models, please don't hesitate to reach out to us. You can contact us at contact@codersarts.com
Comments