Enhancing Prompts with Llama 3 to Generate High Quality Photo
With Automatic1111/ Forge and ComfyUI extensions
Llama 3 has sparked a wave of applications aimed at enriching user prompts. Popular AI generative tools like ComfyUI and Automatic1111/Forge now integrate Llama 3 seamlessly. It making prompt enhancement accessible without complex maneuvers thanks to accompanying extensions.
What is Llama 3?
Llama 3, a large language model (LLM) from Meta. It is available for free commercial use under specific conditions (up to 700 million monthly requests). With versions ranging from 8B to 400B, Meta Llama 3 surpasses comparable models like Google’s Gemma/Gemini, Mistral, and Claude 3 Sonnet, boasting efficient operation on a 4GB GPU.
Installing Ollama
Ollama serves as a facilitator for installing Llama 3. You can download the version compatible with your operating system from the link below.
Download Ollama
Get up and running with large language models.ollama.com
Download Model
After launching Ollama, execute the command in Terminal to download llama3_ifai_sd_prompt_mkr_q4km. Which occupies approximately 4.9GB of storage.
ollama run impactframes/llama3_ifai_sd_prompt_mkr_q4kmThis model combines Llama 3 with 5000 high-quality Stable Diffusion Prompts to ensure superior prompt performance and quality.
You can engage in dialogue with Llama 3 upon download completion. It designed specifically for prompts. If I input 1girl to Llama 3, it will describes the prompt meticulously and providing detailed insights into the concept of 1girl.
Prompts: Adorable, energetic young girl, bright smile, curly brown hair, standing alone in a lush green forest glade, wearing vibrant flower garland around her neck, looking directly at the viewer with sparkling eyes, surrounded by blooming wildflowers, sunlight filtering through trees above.
Installing Extensions
Manually copying prompts each time is impractical, follow the guide for install ComfyUI node and Automatic1111 StableDiffusionWebUI/Forge extension to simplify Llama 3 usage.
Automatic1111 StableDiffusion WebUI/Forge Extension
Both WebUI and Forge versions share the same extension and installation process, as detailed below.
Navigate to Extensions > Available and click Load from, then search for IF_prompt_MKR to find and install it.
Settings of extension
After installation, go to
Settings>Uncategorized>iF_prompt_MKRand set Select the API for generating prompts toOllama.
then in the Script section, select
iF_prompt_MKR.In the Text Model, choose Characters as
iF_prompt_MKRSelect Text-Model as
impactframes/llama3_ifai_sd_prompt_mkr_q4km:latest.You can enter a simple prompt in the input prompt field.
Note that enabling extension will replaces the original prompt box with prompts generated by
iF_prompt_MKR.
Under Styling (Optional), customize and style prompts. Selected styles will display prompts for immediate modification.
In Prefix & Embeddings and Suffix & Loras select embeddings Models and Lora Models if desired. Although these are optional.
Choosing a Negative Prompt will display options in text field. I opted for simple_negative.
I used Juggernaut XL Lightning as the Checkpoint Model. Please adjust steps and CFG scale if not using Lightning or Turbo Model.
When using this extension, Terminal offers prompts guidelines and suggestions for reference.
Comparative Testing 1
In iF_prompt_MKR Text Model, input girl, blue sky background as prompts. Styling choices were Epic and Photographic, with simple_negative as the Negative Prompt.
Generating images with same prompts without Llama 3 support.
While both sets depict a girl against a blue sky background, the images generated with Llama 3 prompts are richer detail and background context. Llama 3 significantly enhances prompts, particularly when original prompts lack specificity.
ComfyUI Nodes
After installing Ollama, use ComfyUI Manager to search for ComfyUI-IF_AI_tools and install it.
And download workflow in following link:
Using Llama 3 generate Prompt for ComfyUI
This is a ComfyUI workflow for using Llama 3 to generate prompt, it also provide a workflow to generate two different…edmondyip.gumroad.com
I configured it to compare regular prompts with Llama 3 enhanced prompts. In the IF Prompt to Prompt section, I usedtoy cat as the prompt.
I selected Juggernaut XL Lightning as checkpoint model, adjusting
stepsandcfgif not using the Lightning Model.Styling embellish prompt set to
Epicand style prompt toDigitalArt, withsimple_negativeas the negative prompt.
In IF Display Text, observe how Llama 3 generated prompts differ from native prompts, with the top section displaying prompts and the bottom showing negative prompts.
Comparative Testing 2
Generating images with same prompts without Llama 3 support.
While both sets depict toy cats, Llama 3 enhanced prompts result have richer and more detailed in background context. However, some images lose the toy-like feel due to their realism, but it still showcasing the significant enhancement Llama 3 offers.
Conclusion
Extensions alongside Llama 3 greatly simplifies the process of enhancing prompts. Although occasional deviations may arise following enhancement, the overall test outcomes are highly satisfactory, particularly when the initial prompts lack detailed background descriptions.
The supplementary context provided by Llama 3 significantly enhances images, rendering it an indispensable element within Stable Diffusion’s prompt generation arsenal.