Page 1 of 1

Sync files slow to receive DDb acknowledgement

Posted: Sun Sep 11, 2016 5:38 am
by AndyAmphlett
Tom,

the new system whereby sync files are imported to the DDb within 15 - 20 minutes of sending, has been working perfectly. I have quickly adopted a workflow, submitting records as soon as they are entered, and then validating them as soon as I get the DDb acknowledgement.

However something has changed over the last day or so. My last two sync files did not receive an automated DDb acknowledgement for several hours, instead of a few minutes. For example last night I sent a sync (with just 13 records made that day). I sent the sync at 21.36hrs, and had the automated reply saying received at by 'Bob Ellis' at 21.39. The DDb acknowledgement arrived at 03.34 the next morning, c. 6 hours later. I don't know if the delay is the DDb sending the acknowledgement, or if record import has slowed.

Andy.

Re: Sync files slow to receive DDb acknowledgement

Posted: Sun Sep 11, 2016 10:34 am
by admin
Hi Andy,

Sorry about the slow imports. There have been database problems over the past couple of days which had caused scheduled DDb tasks to somewhat clog up. The database is split between slow and fast storage space on the server and we'd begun to run out of space on the 'fast' partition. To allow that problem to be resolved I had to temporarily move some of the heavily used tables to the slow partition. It then became difficult to put the moved tables back again, because scheduled tasks were over-running so that there was never a point when the system was not under heavy load.

Things should be back to normal now and the backlog of sync files is clear.

More generally there are occasions when synch files will be slower than usual to import, because the system can run only one import task at a time. Synch files will always be pulled from the hub quickly (on a 15 min cycle) - it is that which triggers the first email notification that appears to come from Bob, but the files are then queued if another import task is already underway.

The most recent BSBI e-newsletter includes a request that all VCRs reset their synch record with the hub and send a new synch file - that may result in a series of large files to import over the next few days.

Tom

Re: Sync files slow to receive DDb acknowledgement

Posted: Mon Sep 12, 2016 11:08 am
by admin
Synch file import may be slow again today (Monday 12th) as I've just started a large (million record) import which could take all day to run and will block MapMate imports until it is finished. Synch files will still be pulled from the hub and queued, so there is no problem with sending them.

Re: Sync files slow to receive DDb acknowledgement

Posted: Mon Sep 12, 2016 10:01 pm
by AndyAmphlett
Tom,

is it ok to send a sync file to the Hub before receipt of an email saying that the earlier sync file had been imported? In the old system, a second sync would overwrite the first. It makes a nice workflow to just enter a batch or records and send a sync straight away.

Thanks,

Andy.

Re: Sync files slow to receive DDb acknowledgement

Posted: Mon Sep 12, 2016 10:18 pm
by admin
Hi Andy,

During the import process the system sends two emails, the first sent via MapMate (ostensibly from Bob Ellis), indicates that the file has been pulled from the hub. Once you've received that then it's safe to send a new file to the hub.

Unfortunately the import delays today will be repeated tomorrow as the large import failed and will need to be retried.

Tom