How to Create a Great Prompt for Stable Diffusion

A popular, deep-learning text-to-image model, Stable Diffusion (SD) allows you to create detailed images based on text prompts. But how impressive and detailed your pictures turn out depends on how specific your text prompts are.

Developing great prompts involves lots of experimentation. In this article, we’ll go through some settings to dramatically change your image variations and how to set up SD to run locally on your PC.

How to Create a Great Prompt for Stable Diffusion

“Prompt crafting” takes time and experimentation to achieve the best results. You should be as specific as possible and acutely define your art styles or mediums and particular artists. Also, avoid keyword jamming.

Next, it’s important to know how to customize the settings of the SD features to automate your testing.

The tips that follow are based on a local installation of SD; however, they are also applicable if you’re running the online version. For detailed steps on installing SD locally, scroll down to the “How to Set Up Stable Diffusion on a Windows PC” section.

Ensure Your Prompts Are Working

Before we start, ensure the prompts are ready to go by doing the following:

1. On the “txt2ing” tab, leave the prompt field empty and scroll down to the “Script” section.
   
2. Click on the drop-down menu and select “Prompt from file or text box.”
   
3. You can drop your prompts text file at the “File with inputs” window. Alternatively, check the “Show Textbox” option and enter your prompts in the “Prompts” window.” It’s probably easier to work with a text file as it’s easier to modify and save.
   
4. Go to the location of your text file and drag the file into the window. If you make changes to the file, you’ll need to drop the updated file into the prompt window, as the UI will not update it automatically.
   
5. At the “Seed” input field, set your custom seed, then click “Generate.”
   

If you’re happy to experiment using the art generated by your prompts, we’ll start by using the Classifier-Free Guidance (CFG) Scale feature.

Experimenting With CFG Scale

The CFG Scale measures how close you want the model to stick to your prompt when generating related images. For example, a CFG Scale value of “0” will generate a random image based on the seed. On the other hand, a CFG scale of “20” and SD’s maximum will create the closest possible match to your prompt.

Follow these steps to experiment with your prompt using the CFG scale feature:

1. Go to the “Script” and then choose “X/Y Plot.”
   
2. Click on the “X type” drop-down menu and select “CFG Scale.”
   
3. At the “Y type” drop-down menu, choose “Steps.”
   
4. At the “X values” input field, consider setting the CFG scale to “3–5.” This will generate whole number versions of your image. If you wish to produce half numbers, enter: “3-5 (+ 0.5) using round brackets.
   
5. Then use the “Steps” in the “Y values” field to define the number of variations between the range. For example, to test out between 10 and 40 steps, enter “10-40.” To use five variations, enter “10-40 [5] with squared brackets.
   
6. For a clear overview, ensure “Draw Legend” is checkmarked.
   
7. Click the “Generate” button.
   

Depending on your requested variations, you’ll receive many render options. In addition, bear in mind that all images are in full resolution, so the version(s) you like will be the finished product.

Experiment With Prompt Matrix

“Prompt Matrix” is another powerful way to test your prompts as you generate more variations from the same prompt. Follow these steps to customize the prompt matrix feature:

1. Go to the “Script” drop-down menu and choose “Prompt Matrix.”
   
2. Enter the prompt in the prompt field, then hit space. Enter a vertical pipe character – “|” – then add another space. Enter the different style versions you want to use, for example, “oil painting” or “watercolor,” and use the vertical pipe to separate each one.
   
3. Once you hit “Generate,” depending on the number of variables entered, that number multiplied by itself will be the number of variations displayed. For example, 4 arguments X 4 = 16 results.
   

Experimenting With Sampling Methods

The Sampling Method refines your image from noise to recognizable shapes. Follow these steps to test out sampling methods:

1. Go to the “Y type” drop-down menu and select “Sampler.”
   
2. In the “Y values” text field, enter the sampler method, e.g., “Euler a,” followed by a comma to separate other sample names. Consider testing in at least three ways.
   
3. To keep things simple, set the “CFG Scale “X value” to three variations by entering “3-5.”
   
4. Hit the “Generate” button.
   

How to Set Up Stable Diffusion on a Windows PC

If you have around 15GB-20GB of free disk space, you can access SD for free by installing it on your PC. The following steps include direct access to all the files you need to install and download.

To make the process easier, consider creating two folders, one to save all your downloaded SD files and the other to install your local version of SD. For example, you can create a folder in “Documents” and call it something like “SDLocal” with no spaces, as SD may have an issue with spaces in the directory name.

Download Stable Diffusion Files

1. First, visit Python.org to download the latest release of Python.
   
2. Scroll down to the “Files” section, then select the Windows installer (64-bit) version to download it.
   
3. Visit the git-local-branching-on-the-cheap “Downloads for Windows” page.
   
4. Click on the “64-bit for Windows Setup” option to download it.
   
5. Now visit GitHub to download the Stable Diffusion web UI. Click on the “Code” drop-down menu, then select “Download ZIP.”
   
6. Go to the huggingface.co website to download the latest SD version. On this web page, you’ll need to create a free account. Scroll down to choose the “…full EMA…” version for a comprehensive version. This is a large file and may take a while to finish downloading.
   
7. Visit the GitHub “GFPGAN” webpage to download the Gen files for the “Face Restoration” feature. Scroll down to the “Updates” section with green check-marked boxes. Download the “V1.3 model” for more natural results; also, download the “V1.4 model” for more detail.
   
8. You can also download “Notepad++” from the Notepad++ website. Click on the most recent 64-bit version listed at the top.
   

Install Stable Diffusion Files

1. First, go to your Phyton installation file, double-click on it, then select “Install Now” in the popup window. Click on “Close” once the installation is complete.
   
2. Go to the “Stable Diffusion Web UI Master” zip file, double-click on it, then double-click on the folder version. Select all the files inside that folder, then move them into the SD local folder you created earlier in the “Documents” folder. Ensure all the files are visible there.
   
3. Go back to your folder containing all the downloaded files. Find the “GFPGANv1.3.pth” and “GFPGANv1.4.pth” files, then drag them over to join the other SD files in your SD local folder.
   
4. Find the “sd-v1-4-full-ema.ckpt” file from your downloaded files folder, then click on the file name to change it to “model.” Drag it into your SD folder.
   
5. Double-click on the Git .exe file, agree to the license, then keep clicking “Next” until you get to the “Installing” screen, then click “Finish” once it’s done.
   
6. Find the notepad++ installer .exe file to install it then click “Finish”.
   
7. From your SD local folder, click on the “webui-user.bat” file, and a “Windows protected your PC” popup will display. Click on “More info,” then “Run anyway.”
   
8. The command prompt window will open and display a “Couldn’t launch python” error. Leave the command prompt window open.
   

To clear this error, you’ll need to essentially connect Python to SD; here’s how:

1. Right-click on the “webui-user.bat” file, go to “Show more options,” then select “Edit with Notepad++.”
   
2. Notepad++ will open. Click on the search icon and enter “Phyton.” The Phyton app will display the results.
   
3. Right-click on the Phyton app and select “Open file location.”
   
4. The Phyton folder will display; right-click on the Phyton 64-bit shortcut, then select “Open file location.
   
5. Right-click on the “phyton.exe” file, then select “Copy as path.” The path will be copied to your clipboard.
   
6. Go back to Notepad++, and place the cursor next to the “set PHYTON=” line to copy the path. Click on “Save.”
   
7. Go back to your SD local folder, then double-click on “webui-user.bat” file.
   
   • This process may take a while and won’t show you an indication of progress. Once the installation is complete, the command prompt window will be filled with SD information. This window needs to remain open when running SD locally.
8. The most important information is the IP number located at the “Running on local URL:” line towards the bottom of the screen.
   
9. Copy the URL, then paste it into your browser address bar to access the local version of SD.
   

The Art of Creating a Great SD Prompt

When it comes to working with AI art models in SD, the best images are created from specific and well-worded prompts. There are even great sites that let you generate images for stable diffusion online. But prompt crafting takes time and a lot of experimenting with the features to see how SD responds to particular tweaks. The more time you spend playing around with SD, the more you realize that getting it to generate your desired art is an art unto itself.

Have you managed to create images with SD that you were impressed with? Tell us about some of your favorite art in the comments section below.