Generating synthetic data step by step in Blender

Sometimes in deep learning there are tasks for which it is not possible to find open datasets, and it is difficult and expensive to collect data. Typically these are very specific tasks with the requirements of a specific customer. In my practice, this situation happened at least twice.

The first time it was the task of detecting traffic signs for a competition of self-driving vehicles. In this task, it was important to learn how the signs look like in the competition (https://www.youtube.com/watch?v=BG68Hz1_T28). The second time it was the task of image enhancement for hieroglyphs. In particular, photographs of books and instructions with hieroglyphs under different lighting and at different angles.

Both tasks described above are ideal for generating synthetics for them using Blender. They do not require the modeling of complex objects, which means there will be no problem with the fact that they look unrealistic. In addition, it does not take much time to create such synthetics from simple geometric shapes.

In this article, I decided to share my experience of creating synthetic data in Blender using step-by-step instructions. Further, we will consider in detail the creation of synthetic data for the task of image enhancement of hieroglyphs. I hope this article will be useful for data scientists and make it easier for them to collect more data for training models.

Scene creation

First of all, go to Blender and create a new project:

After creating a project, you will see a workspace with a basic preference — a cube. It can be deleted, to do this, select it with the mouse and press Delete. Next, we want to create a plane to which we will apply a texture (for example, turning it into a book page):

The next step is to position the camera above the plane. This can be done in the panel on the right:

Now prepare the textures that we will apply to the plane we have created. For the task of image reinforcement of hieroglyphs, I used screenshots from PDF in Chinese. In this case, I have collected about 150 images. What textures are needed and how to find them is a creative process and depends on your task.

So let’s add texture to the plane we created earlier. To do this, select a plane in the workspace and in the panel on the right go to the Materials tab. And create a new material:

To add a texture as a material, expand the panel next to Base Color. Select Image Texture from the dropdown menu. After that, in the panel on the right, you can specify the path to the image by clicking on the Open Image button.

In order to see the loaded texture in the workspace, activate the preview of textures and lighting:

Script writing

So by now we have a scene with one textured plane, a camera and a standard light source. Now you need to write a script that will replace the texture and vary the scene parameters (lighting, plane position). Blender has a built-in Python interpreter with which you can control the editor. To write a script, go to the Scripting tab on the top panel. Blender will switch to Python scripting mode:

In this mode, the screen will be split into a 3d scene editing area, a script editing window, a Python console and a message output window. To create a new script in the text editor area, click Text-> New. The created empty script can be saved by the keyboard shortcut alt + s with the extension .py.

Now you can start writing code. The first step is to import all the required packages:

bpy — to work with Blender, glob — to get all image files from the specified folder, random — will be needed to vary various scene parameters.

Then we need to get the paths to all images using the glob function. We will also create folders for the resulting renders. Since we are generating data for the image enhancement task, we will need two folders for Low quality and High quality images:

The next step is to set the render parameters and get references to the objects that we want to manipulate:

With the N_aug parameter we will control how many different positions of the plane and light source we want to use for each texture.

Now, as we iterate over all the images in the folder, we’ll be using them as textures. To do this, we will write the path to the image in the data.images[-1]. Since there is only one image in the scene, which we added to the plane, in this way we will change it:

For each texture, we generate N_aug different scenes. For each scene, we vary the position of the light source and the angle of rotation of the plane along two axes.

Now the resulting scenes need to be rendered with two different qualities. In our case, low quality images should be slightly blurry and out of focus. To achieve this effect, we will vary the focal settings of the camera. To save LQ images, we will enable DOF, and disable it for HQ.

As a result, we got the following script, varying the lighting conditions and the angle of the plane:

Conclusion

The above script allows you to get synthetic images in Blender as in the example below:

Blender is great if you need to quickly generate synthetic data. It is doubly good if your data can be represented in the form of simple geometric objects, as in this article. In addition, the built-in Python interpreter makes it easy for data scientists to get started with Blender, as it is usually our main working language.