This week we are going to look at how Reality stores its data. Since version 1.0 Reality integrated with the host application, Studio in this case, so that no external file is needed for its own data. This is a key concept: you load your Studio scene and, when you call Reality, all the data about materials, lights and cameras are there.

Reality 3 has a lot more data and much of it is quite complex. Last week we saw that Reality 3 has now around 15 textures while the previous version only managed 2. This makes the job of storing and retrieving data a lot more complex and delicate. Also, one of the goals for Reality 3 was also to adopt the native Studio format instead of simply injecting its own “blob” of data in the Studio scene.

Studio uses a file format that is called DSON, which is a variant of the Internet standard JSON (JavaScrip Object Notation). JSON is a great format that is used by many applications to exchange data. Storing data is one things that almost every program needs to do. It’s important to remember that a file is simply a “bucket” of byes, with no implied structure in it. This means that every program needs to define its own data structure. That is another task in the hands of the programmer, who needs to define the logical structure of the data file. We also need to anticipate possible expansions of the program that will need to be handled without breaking compatibility with the past. If you used Microsoft Word for more than a version you know how easy is to fail that task 🙂

Most of the data that we need to store in a file is usually in the format KEY = VALUE. This menas that we generally have a parameter name, the key, and its value. For example, if you set the rendering resolution to 1920 by 1080 the file will have something like:
width = 1920
height = 1080

In JSON we write the key between quotes and we separate the value with a “:”. For example:

"author_name" : "Paolo Ciccone",
"author_site" : "",
"app_version" : 3.5,
"send_email"  : true,
"gamma" : 2.2,
"figures" : ["V4", "Dawn"],
"scene" : {
  "usesAlpha" : true,
  "numThreads" : 8,
  "width" : 1280,
  "height" : 720

Each value can be one of the following:

  • A string between quotes. For example: “Reality”
  • A number, either integer or floating point
  • A boolean value. Either true or false
  • A list enclosed by square brackets
  • An object, enclosed by curly brackets. An object follows the same KEY : VALUE syntax

JSON is a great format that can be easily read by any program and that is still readable by humans. Compared to XML it has the advantage of being a lot simpler to read and write, and to take a fraction of the space required. Kudos to the Studio developers for adopting it.

When I started writing Reality 3.0, initially for Poser, JSON was adopted as a storage format. If you select a a material with Edit | Copy in Reality 3 and then paste it in a text editor you will see something like this:

Reality3CD { “alphaMap” : ““, ”alphaStrength“ : 1.0, ”bmNegative“ : –0.001, ”bmPositive“ : 0.001, ”bmStrength“ : 0.5, ”channels“ : { ”Bm“ : ”“, ”Dm“ : ”“, ”Ka“ : ”Default.Ka“, ”Kd“ : ”Default.diffuse“, ”Kg“ : ”“, ”Kl“ : ”“, ”Ks“ : ”Default.Ks“, ”Kt“ : ”Default.Kt“, ”alpha“ : ”“ }, ”coatThickness“ : 0.2, ”dmNegative“ : –0.01, ”dmPositive“ : 0.01, ”dmStrength“ : 1.0, ”edited“ : false, ”emitsLight“ : false, ”innerVolume“ : ”“, ”isNormalMap“ : false, ”keepSharpEdges“ : true, ”lightGain“ : 1.0, ”name“ : ”Default“, ”outerVolume“ : ”“, ”smoothness“ : true, ”subdivision“ : 0, ”surfaceFuzz“ : false, ”textures“ : [ { “color” : “#ffffff”, “dataType” : 0, “name” : “Default.Ka”, “type” : 1 }, { “color” : “#ffffff”, “dataType” : 0, “name” : “Default.diffuse”, “type” : 1 }, { “color” : “#262626”, “dataType” : 0, “name” : “Default.Ks”, “type” : 1 }, { “color” : “#ffffff”, “dataType” : 0, “name” : “Default.Kt”, “type” : 1 }, { “color” : “#ffffff”, “dataType” : 0, “name” : “Default.Kd”, “type” : 1 } ], ”topCoat“ : false, ”translucent“ : false, ”type“ : ”Glossy“, ”uGlossiness“ : 1200, ”useMicrofacets“ : false, ”vGlossiness“ : 1200, ”visibleInRender” : true }

Don’t panic! That is just a “blob” of JSON text that has not being formatted. Computers don’t care about formatting and so Reality does not format the data that is stored internally. Doing so saves space. On the other hand, we do need things to be nicely formatted in order to be able to read the information. That is very easily done. Let’s go to the JSON Lint website and paste the text in there making sure to avoid the Reality3CD portion. We only select the text between the initial and final brackets. We click on the Validate button and we get this:

Reality for Studio - JSON Lint

So, as we can see Reality uses JSON as the internal interexchange format for its data. This was part of the initial design since I wrote the first line of code for Reality 3. It’s important to highlight this. Reality 3 was first released for Poser, because Reality had been available for Studio for more than two year, but the design always evolved with both applications in mind.

So, in the past few days I have been busy implementing the saving of Reality data inside the Studio scene files. My goal was to integrate it with the Studio format so that the Reality data chunk looks completely native. I want to thank Rob at DAZ for his clear explanation of the Studio plugin interface that makes this possible. So, I’m glad to report that the goal has been achieved. Let’s take a look.

When we save a Studio scene we generate a .duf file. That is a text file in DSON/JSON format. Studio compresses it using the gzip format, which is not the same of the zip compression. On Mac OS we can uncompress it using the command from the Terminal window:

gunzip -kS .duf [fileName].duf

There is probably a similar command for Windows.
Of course [fileName] needs to be replaced with the actual name of the file. This will give us a version of the DUF file that is uncompressed. We can now open that file in a text editor and take a look. The following image shows a small portion of the Reality data, formatted to fit the screen, embedded directly inside a DUF file:

Reality - scene data

As you can see the everything is in clear and easily readable .

To recap, at this point Reality is able to convert the materials from Studio, it can edit and change any material, it has a very nice material preview and it can save the data in the Studio file. The next task is to make it able to read the data back and restore everything.

If you are not yet a subscriber of this blog please consider becoming one. By subscribing you will receive automatic updates about Reality.

Happy rendering.

Pin on PinterestShare on FacebookTweet about this on TwitterShare on Google+