 |
| (Image: InstructPix2Pix) |
If you’ve ever marveled at AI image generators like Dall-E or Midjourney and wished you could upload your photos for editing with just text descriptions, then you're in luck. A new AI image editor tool called InstructPix2Pix can do just that. Available on the AI tool website Hugging Face, InstructPix2Pix requests an input image and prompt instructions and produces an edited image with the desired changes.
In order to obtain the training data for the AI tool, its creators leveraged the knowledge of language models GPT-3 and Stable Diffusion to create a vast dataset of image editing examples. This dataset was then used to train InstructPix2Pix. However, unlike Stable Diffusion, which is an image generation model (text-to-image), InstructPix2Pix is an image editing diffusion model.
 |
| (Image credit: Express photo) |
On November 17, 2022, Tim Brooks, Aleksander Holynski, and Alexei A. Efros published a paper on InstructPix2Pix several days before the launch of ChatGPT.
How to use InstructPix2Pix to edit photos:
To use InstructPix2Pix, the easiest way is through its Hugging Face web app. Users upload the image they wish to edit and add their desired edit instructions to the text field. The output is then generated, but the process can take up to ten minutes, so patience is essential. However, considering the amount of time that manual editing would take, the wait is worthwhile.
Testing InstructPix2Pix out:
Our initial experience with InstructPix2Pix was disappointing. We uploaded a scenic picture of Noida's skyline with the prompt "replace the buildings with mountains." The output was highly distorted, and the buildings were barely replaced, if at all.
 |
| (Image: Zohaib Ahmed/Indian Express) |
 |
| (Image credit: Express photo) |
Adjusting the CFG and Text CFG weights to 8.5 and 1, respectively, produced better results. The buildings were replaced by a neat mountain range, with only the yellow-black road divider looking off.
 |
| (Image credit: Express photo) |
We also tested InstructPix2Pix with an image of a white cat, feeding the prompt "change the cat's color to black" to the tool.
 |
| (Image credit: Pixabay) |
 |
| (Image credit: Express photo) |
InstructPix2Pix did a good job with this one, replacing the cat's white fur with black while retaining the white whiskers.
Conclusion:
In its current state, InstructPix2Pix is a great AI tool to experiment with. While it may not produce images as convincing as Midjourney's viral Pope in a Balenciaga jacket photo, it is still a glimpse into the future of photo editing. It is possible that someday, complex editing procedures will be replaced by simple text-based prompts.
0 Comments
If you have any doubt, Please let me know