Plus Feature Request: The export file path is always static. I'd like to have the possibility to create a dynamic file path with properties like File path = C:\Server\PublicFolder\Ek_Export\{CUSTOMER}

Today I use this function in a workflow, but the problem is that the export for every workflow is sent to the same folder.

Cheers

Daniel

Hi Steve,

Except for the standard limitations, there are no specific ones tied to EK.

Kind regards,

Mihovil

Hi Mihovil

We need a solution here. I get responses from people that they can't use our created "Export-Workflow" because it only can export into one fixed folder AND it needs way too much time.

We are talking about 11 Minutes for one document. After the process was finished it was 14 hours for 65 Files / 43 MB.

Export via client a thing of a second / copy & paste.

I think there is a problem. Could you please check?

Cheers

Daniel

Hi Daniel,

Yes, I can understand that performance issues can make use of this module not as effective. I've tried to replicate Export functionality and I've found that there aren't performance issues even with very large files (over 30 MB). Of course, my configuration and environment cannot be the same as yours (or any other user experiencing these issues), but it proves that Data Transfer can work fast even with large files.

So, I've tested two scenarios:

1. User has 3 large DOCX files (Class: Document) and wants to export all three into a folder on his C: drive

 - All three files are saved to local folder under a second

2. User has 3 large DOCX files (Class: Document) and wants to export specific file when that file enters Export state in Workflow

- Specific file is saved to a local folder under a second as well

Note: for testing purposes you can download large DOCX from https://www.learningcontainer.com/sample-docx-file-for-testing/

You can try to recreate those scenarios on freshly set up vault using default Document class, with latest officially released EK (5.0.0). If you still have performance issues, I would suspect that there is a problem outside EK. Or, if you want to test it on existing environment, you can use said rules on Document class provided it has the same structure as default.

If the performance issues persist I would suggest restarting the server and the vault or trying to run Data Transfer on objects that are not being affected by other EK rules.

Regards,

Mihovil

Hi Mihovil,

I'm not sure if we've understood each other correctly. The problem isn't that we're using large files, but the number of documents processed in the export. We are conducting three exports simultaneously, and this process has already been ongoing since last Friday.

Please check the JSON configuration file again. I would also like to suggest that we look at this together in a Teams session. This way, the technical situation will surely become clearer for you, and we can make progress more efficiently and satisfy our mutual client. This function is essential for us, and we urgently need a solution to this problem.

Best regards and thanks in advance
Steve

Hi Steve,

I've revisited your original post and I have some comments.

Firstly, all three rules have the same trigger, which is not uncommon. The only thing to keep in mind is that when the object of class "CL.GF.Export", and the workflow "WF.Export" enter the state "state.Export.StarteExport", all three rules will be set to execute in background in one batch, in the order in which they have been defined in the module, so:

#1: OT Geschäftsfall: CL Export - Revision / Buha-Akt
#2: OT Geschäftsfall: CL Export - Revision / Bankbelege
#3: OT Geschäftsfall: CL Export - Revision / Beilagen

A potential issue can occur when there are consecutive triggers occurring; in that case, multiple batches of said rules will be run, and each rule will overwrite a previously exported file.

This increases the total time of export until the last rule finishes its course.Please check if there are multiple triggers occurring in your case. If so, I would suggest that you set the parameter "Overwrite" to "No" in Export settings. This will ensure that multiple instances of running rules skip saving the file if it already exists. If you need this option, do not change it.

Secondly, "OT Geschäftsfall: CL Export - Revision / Buha-Akt" has very loose search conditions for export. For example, it doesn't have Class condition, which means that there can be many files that will be exported regardless of class. And if so, it will delay the execution of the other two rules.
Please check if that condition is missing in Export settings - Objects to Export - Search - Property conditions.

You mentioned that export is running for days now, could you screenshot the "Data Transfer" and "Asynchronous operations" sections in EK Dashboard and attach them?
Also, how many files are you wanting to export (approximately) for each rule? 
Could you browse for logs on your server and attach them for further analysis?

For testing purposes in a non-production environment, I would suggest configuring these rules as "Run once", removing Trigger Source completely and running them through the Dashboard individually. That way, the rule will be triggered only once.

Kind regards,

Mihovil

Hello Mihovil,

Thank you for your response.

Indeed, using the same trigger for three exports is not unusual and in this case, it is necessary and required. As soon as we can use a Property-Setter with this module, we would cascade these exports and use different triggers for each.

Overwriting the files is not possible, as each export uses its own directory. Therefore, this could not be the cause of the problem.

The search conditions for the export "OT Business Case: CL Export - Revision / Buha-Act" are correct and also necessary. When I search manually with these conditions in M-Files Desktop, I immediately get the desired results. Therefore, I exclude search performance as an issue on my part.

The export was completed yesterday. However, I have just started it again (run once), and the log shows that only every few minutes a file is written to the file system. See an extract from the log Line 1&2 at 10:40 AM, then Line 8 at 10:46 AM. What is the reason for this? The export started at around 10:40 AM. By 11:27 AM, only 9 files were exported.

As I have already offered, I suggest we look at the problem in a joint Teams session. I feel that this way we could find a solution more quickly.

Attached is an extract from the log and the requested screenshots.

Thank you in advance.

Best regards
Steve

Hi Steve,

We are checking this issue with our development team and we will post our findings as soon as possible.

Kind regards,

Mihovil

Hi Steve,

We are still working on that, for now it seems that the problem lies in multiple rules triggered at once for that same object.

Could you please provide us with some additional information,
- Is performance good if you trigger these rules one by one?
- What is the approximate number of documents that these rules export?

Kind regards,
Tadej

Hey Tadej,

Posted by: @daniel-mislovic

↑

We are talking about 11 Minutes for one document. After the process was finished it was 14 hours for 65 Files / 43 MB.

For the second Question, see above. First one will Steve answer later on.

Cheers

Daniel