Getting Enormous Ticket Data Gracefully

Hi,

I have enormous amount of tickets data in my source Freshdesk instance and I am using the List Tickets to extract this data. However I find that this API is very limited as it only provides 300 pages of response. Also I see that the pages returned are having data in descending order of created date. Hence I am only able to retrieve 300 pages of most recently created tickets. I am not able to get very old tickets due to this issue. Are there any alternatives?

I see the Query Ticket API, but the API can only provide 10 pages at a time and looks very trivial for extracting millions of tickets.

Thanks in advance!

Hi @Cvam

There is no easy way to do it in a reasonable time via the API due to the rate limits.

We exported over 1mill ticket records for a customer this using the export feature in the UI which will send you a csv file of the data. But, that won’t give you everything the API does.

We solved that by also doing a XML export and then we wrote a node.js script that merges the CSV and XML in to a set of new CSV files with some pretty rich data in them, including some stuff you don’t get from the API which created using data from other API endpoints.

We then created a new custom app that acts like a webhook which syncs ticket updates as well as new data to the system where we exported the initial data. It was done to create a reporting data warehouse for Power BI reporting so that the customer could merge Freshdesk data with finance data.

We have a Power Apps solution we can provide if that’s of interest.

https://appsource.microsoft.com/en/product/dynamics-365/essencientltd1646293568296.freshdesk-power-bi-reporting

Let me know.

2 Likes

Wow @RobAtOpinyin
Thats pretty impressive.
Thanks a ton for the info!

Let me see if I can capitalise on this info

-Cvam

1 Like

Hi @Cvam,

Yes, the List Tickets API has a limitation of only returning up to 300 pages. If you would like to export all the tickets, it’s not possible to implement via API.

You can export the details via a file manually and use this in another system to import if it supports or write a transformer on your own to comply with the format the third-party support.

Steps to get the file export of all the tickets:

  1. Log in to your Freshdesk account as an administrator.
  2. Go to the “Admin” tab in the top navigation bar.
  3. Under the “Account” section, click “Account Details”.
  4. Click the “Export” button to initiate the export process.
  5. Freshdesk will generate the export file and email you a download link when ready.
  6. Check your email inbox for the export completion notification.
  7. Follow the download link provided in the email to download the exported file. It will be in an XML file for each entity in the account zipped into one file.

Using the Freshdesk Data Export feature, you can bypass the API pagination limitations and retrieve all tickets. Remember that the export time may vary depending on the volume of all data in your account, so you might need to wait for the export to complete.

1 Like

This topic was automatically closed 6 days after the last reply. New replies are no longer allowed.