Late delivery issue for 2023-01-22 and 2023-01-23

Resolved
Resolved

We have resolved the late delivery of flat files and it is now operating as usual.

Recovering

We saw that the files for 2023-01-22 and 2023-01-23 could be delayed and currently we are processing the backlogs while working on resolving the core issue to enable delivery of all files up to T+12h.

Please note that T+12h is not a hard/strict deadline. After that date, we can still: (a) reupload the files if we can improve their quality, (b) upload the files for the new symbols that were not listed on our platform at the time of the upload (backfilling the historical data), (c) not be able to upload the data until T+12h because of the congestion or ratelimiting from the Google Cloud Storage API side.

Kindly monitor the bucket inventory and process the new/modified objects periodically. This way the pipeline would be as much as possible flexible to the time of the upload.

We apologize for the inconvenience and thank you for your patience.

Began at:

Affected components
  • Data delivery
    • Download
    • Upload to S3
    • Upload to GCP