guido@nextnext.nl Hey Guido, sorry for the very late reply.

There isn't an explicit mode to only load new data rows (versus the original rows), however, the way importer creates nodes and relationships using statements is idempotent, so if you load the same set of files twice or repeatedly, you will end up with the same result.

Therefore if you drop a set of new files on that reflect new data, or append rows to a file you will only get the new data added in both cases if you load those file.

The only small subtlety to be aware of is how we use IDs to set properties. If you have two rows that have the same ID, but differing row values that you set to properties on that node, the properties of the last row to be read will be written and "win".

Hopefully that is helpful, let me know if you have any more questions and once again apologies for the long wait.

Greg

Gregory KingHi Gregory, my turn for a late reply.

I have been looking for a solution and I might have found one.

Since I want to keep all existing data in the database intact and only want to add new data (with new IDs) by importing csv's, I can upload csv's with only new data in Workspace Importer. Some of these csv's, however, would be empty (only column headers) and no new data will be imported.

But I just want to be sure of one thing.

When I run a preview on a node corresponding to a csv with new data, I get to see only the new nodes, that looks okay.

But when I run a preview on a node corresponding to an empty csv, I get to see all the exisiting nodes in the database. And I want to see no nodes at all since I am importing nothing.

Does that mean that empty csv's do not overwrite the previous ones (and I would lose all modified data in the database) or is this "just a glitch" in the preview?

Thanks!

Hey guido@nextnext.nl, behind the scenes the way preview works is by running an uncommitted transaction into the database you are connected to. This means preview will never write any data, but it does mean it has the potential to match into data that already exists in the database. That means that when you're doing a preview on a non-empty database, if the relationships you are previewing find matching IDs in the database, you will see them in your preview even if they aren't in the set of files you are previewing. Hopefully this quirk makes sense.

FWIW, I think importer requires you to have at least one line of data in your files, so completely empty files with just a header likely won't work (but you could retain a row of already loaded data as a workaround). I hope this helps.

Gregory King, thanks, quite reassuring.

While testing very carefully, I found that completely empty files are indeed rejected. But a single space on the first row -or an existing row, as you suggest- does the trick.

Nice workaround.

An option "overwrite all data/only add new data" in Importer would be a nice idea for the Suggestions Box, though.