file format

[toro_rossoParticipant]

hello,

I was wondering if there were any plans to support a diff-able & merge-able file format for NodeCanvas?

right now, the tool serialize to a single line json, which is impossible to diff or merge.

since json is a subset of yaml, maybe you could embed the pretty json inside the .asset file?

or simply serialize in yaml?

thanks

[GavalakisKeymaster]

Hello and welcome.

It is a weird situation really and this is not the first time this topic has been discussed in the forums.

The problem is that even if I serialize the string with prety json format, Unity will still remove all spaces and paragraph characters, thus effective removing the prety json formating. I have tried a lot in the past to force the serialization into staying as a prety json instead of being re-formatted to a single line so that it is easy to diff in source control, but to no avail :/ No matter what I try, Unity will re-format the string into a single line.

It is certainly something I want to somehow solve if at all possible, considering of course that the serialization will need to remain in json data format and can not really be changed at this time, mainly due to maintaining backwards compatibility, but I have not yet found a solution unfortunately due the reason stated above.

Thank you.

[toro_rossoParticipant]

hi!

thanks for replying!

i’m new to unity, so I dont know what is possible or not.

but would it be possible to allow a mode where you store in a pretty json outside of the .asset?

thanks

 

[GavalakisKeymaster]

Hello again 🙂

If you mean exporting to prety json in a txt format, then this is already possible through the Graph Editor window “File” Menu -> “Export To JSON”. This JSON export however is not in sync of the graph, but rather just a way to export/import the graph to a text format.

Is what you mean something similar?

Thanks!

[toro_rossoParticipant]

hi,

I already knew about the export json.

i’m thinking using a TextAsset to store the data instead of string _serializedGraph;

this way, you can specify a custom json format, unity shouldn’t try to compact it.

[GavalakisKeymaster]

Hey,

Thanks for the follow up.

Syncing the serialization to a TextAsset could be possible as an option and it should work. I can take a look at adding this as an option to reference an external TextAsset wherein the serialization string will be stored (in prety json of course). The way I foresee it, it will basically write to that file on every graph serialization callback. Due to backwards compatibility though (and probably for extra safety as well), the json string will also be serialized in the graph object as it does right now, so this might create some redundancy. Of course, this way, there will also be two files instead of one (as far as asset graphs go at least), but only the actual graph asset will be critical and the other (the Text Asset) will only exist as an extra and no problem will be issued even if that is deleted.

Let me know what you think.

Thanks!

[sugoidevParticipant]

I approve of this very much. Having external text assets is nice.

 

One thing I think should be taken into consideration is an option to enforce which file is the canonical one. If I change a text file externally, I would expect the serialized data I changed to be used by NC when the data is loaded into Unity.

[timvParticipant]

This is on my wish list as well!

[GavalakisKeymaster]

Thank you for your input everyone.
@sugoidev , my initial thought was that the TextAsset act only as a reference and be a copy/mirror of the original serialization (stored in the graph), so the canonical serialization always be the graph serialization instead of the Text Asset. This will make things easier (as in less prone to bugs and data loss in case TextAsset gets deleted for whatever reason), as well as avoid extra options in inspectors (which I am never fond of :P).
Do you guys thnk that having a “Serialization Source” option (choice between internal json or external TextAsset) would be important and helpful?

Let me know.
Thank you 🙂

[sugoidevParticipant]

@Gavalakis fairly important for me.

In particular, this would make it massively easier to deal with merging in source control.

I can just look at the pretty-printed text and merge the changes with confidence, then NC will pick up the changes when Unity refreshes.

 

The non-selected “Serialization Source” could be used for cases when we have trouble. If the merge broke the data from the text asset, NC could use the internal data and show a warning about replacing the broken data in the text asset.

[pbpfParticipant]

Hello,

We’ve had the same issue (difficult to resolve conflicts in the single-line JSON embedded in the YAML of the FSM asset file).  I think that having the serialisation source option to pick between a TextAsset or storage internal to the FSM asset sounds reasonable, but only if the _serializedGraph field in the FSM asset is empty if using the TextAsset as the source.  If you continue to serialise to the internal field (as a copy) as well as the TextAsset, that would mean that you would still get conflicts that git / perforce / etc. would find difficult to resolve automatically in the FSM asset file, while the TextAsset with pretty-printed JSON might well be resolved automatically.

I can see the benefits of keeping the default option as an internal serialisation in the FSM asset (e.g. the Unity object references are serialised in the same file as the indices into that array making it harder for things to be out of sync), but it would really be useful to be able to have something that merges cleanly without a lot of extra effort.  Keeping internal serialisation as the default would also ensure backwards compatibility with existing assets.

[pbpfParticipant]

I’ve made this change in the project I’m working on for non-bound graphs, and it seems to work pretty well.  The only things I came across that were not discussed above were that:

• The pretty-print serialiser, since it uses a default TextWriter, will write different line endings on different platforms which is annoying in teams working on Mac and Windows (so I’ve changed that)
• The contents of TextAssets are not writable via the TextAsset – only by looking up where that is on disk, which means that it gets saved immediately, rather than when the project is saved.
• The path to the TextAsset needs to be stored too for use in the editor, since looking it up in the AssetDatabase is not possible when deserialising / serialising

I’d be happy to post a patch here or send it via email or similar, but I’m not sure what you would prefer / permit.

[toro_rossoParticipant]

hello pbpf,

I’d be interested in checkin out your patch. it could even give gavalakis a head start on this feature!

 

 

[pbpfParticipant]

Here you go.  As I say, I only did non-bound graphs (since that was what I needed), but something similar could work for bound ones too.

[Author]

Posts