# Using the WP Queue to Copy REST API Data to Files **Date:** 2019-01-29 **Author:** Dan Maby **Categories:** Development > Recently Matt Shaw from Delicious Brains published a post about a new library they created to help in one of their products. This library WP_Queue provides a Laravel-like Job management system for WordPress. A job queue is a system that allows you to schedule jobs to run in the future. We tend to use jobs … [Development](https://blue37.com/blog/category/development) [View on blue37.com](https://blue37.com/blog/2019/01/using-the-wp-queue-to-copy-rest-api-data-to-files) --- Recently Matt Shaw from Delicious Brains [published a post](https://deliciousbrains.com/email-queue-ses/) about a new library they created to help in one of their products. This library WP\_Queue provides a Laravel-like Job management system for WordPress. A job queue is a system that allows you to schedule jobs to run in the future. We tend to use jobs for two reasons. First, we may need to wait awhile, like if we want to schedule a follow-up email in a week. The other reason is performance. Maybe we need to do something computationally expensive and don’t want the user to wait. A job manager gives us a scheduling system — some way to store jobs until they need to be run — and a job runner — some tool for running the jobs. WordPress’ wp\_cron sort of fits this description. However, through using WP\_Queue, I’ve found it fits my needs better. ## The WP\_Queue The WP\_Queue  package looks great for a few reaons. First, is that the jobs are abstracted from the runner and scheduler. I can write a job class and test it as unit in isolation. Secondly, the jobs scheduler is abstracted. By default, jobs are recorded in the WordPress database, and then when they are needed, they are scheduled with wp\_cron. But I can also use a development driver that makes them synchronous and there is a Redis driver in progress. So let’s get started. ## Set Up The WP\_Queue library is a composer package. First, install the package in your plugin: View the code on [Gist](https://gist.github.com/Shelob9/1d8756c30330d291009e52a51c59a40e). You will need to add the database tables for the scheduler. [The Readme for the package has instructions](https://github.com/A5hleyRich/wp-queue#prerequisites). Add this to your plugin’s activation hook or wherever you add your own tables. ## Creating Jobs If you’ve ever used Laravel’s job queue, the structure of the WP\_QueueJob class, will be familiar. Your job class has to have a handle() method. That method is called when the job is run. You probably will have a \_\_construct() method as well. The key concept to understand about these classes is that the properties of the class are serialized to the database when the job is scheduled and used to instantiate the class when it is run. For example, let’s say you wanted to create a job to run whenever a post is saved. In the \_\_construct() method, you would pass the post ID and use that argument to set a property. When the handle method runs, that property will be set with the post ID. In the handle method, you can use that property — the saved post ID that was saved with the job — to get the saved post from the database. Here is what that looks like: View the code on [Gist](https://gist.github.com/Shelob9/1d8756c30330d291009e52a51c59a40e). ## Copy A Post’s JSON To A File When It Is Saved What we’ve done so far is a job that gets a post from the database. That’s cool, we could start a runner, schedule this job to run every time a post is saved, and make this job get the saved post’s REST API response and write it to a JSON file. Let’s work through that list backward. Why that order? That last requirement will be one class, which I can test in isolation, and if it works, then I can setup the runner and hook into the save\_post action knowing that job works. If I did it the other way around, I wouldn’t know if my problem was the job, or the queue. That’s bad science. [I’ve written about a similar appraoch to running jobs after a post is saved before with Torque using a different task runner](https://torquemag.io/2016/01/use-asynchronous-php-wordpress/). ### Write A Post’s JSON To A File Let’s keep working on our job class. We have everything we need to get the post. We need to get the response object that the WordPress REST API would create for the post, serialize it to json and write that JSON to a file. [I wrote a post for Torque awhile ago about how to get posts from the WordPress REST API without making HTTP requests](https://torquemag.io/2017/04/using-the-wordpress-rest-api-without-an-extra-http-request/). I stole most of this code from there: View the code on [Gist](https://gist.github.com/Shelob9/1d8756c30330d291009e52a51c59a40e). This is the complete job. I used the WordPress REST API’s post control to create a WP\_REST\_Response and used json\_encode() to make it a string and saved it to a file named for the post slug. Where you stored the file and what you call it depends on your needs. Right now, we can instantiate this class to test if it works. We can run it directly with something like this, where $postId was the ID of a published post: View the code on [Gist](https://gist.github.com/Shelob9/1d8756c30330d291009e52a51c59a40e). Then you should see in the path you set for writing a file, a file with the JSON representation of the post. We can also write an integration test for it: View the code on [Gist](https://gist.github.com/Shelob9/1d8756c30330d291009e52a51c59a40e). ### Scheduling The Job To Run When The Post Is Updated Now that we know our job would work if it was scheduled to run, we need to schedule it to run, whenever a post is saved. The save\_post action fires when a post is saved, so we can use that. In the callback function, we will instantiate the job class and pass it to wp\_queue()->push(). View the code on [Gist](https://gist.github.com/Shelob9/1d8756c30330d291009e52a51c59a40e). That’s enough to schedule it to run, as soon as possible. #### Delaying Job Execution The second argument of the push()  method is the delay time to run the job. The last code example didn’t use that argument. The job will run as soon as the queue gets to it. If we wanted to delay it by 5 minutes, we could pass 600 — 5 minutes in seconds — as the second argument. View the code on [Gist](https://gist.github.com/Shelob9/1d8756c30330d291009e52a51c59a40e). ### Setup The Job Runner Last step: we need to setup the queue to run. We can run the queue using wp\_cron: View the code on [Gist](https://gist.github.com/Shelob9/1d8756c30330d291009e52a51c59a40e). When you are doing local development, it’s better to use the “sync” driver, instead of the database driver so jobs run synchronously, IE right away. This makes testing easier. View the code on [Gist](https://gist.github.com/Shelob9/1d8756c30330d291009e52a51c59a40e). ## Use Without WP-CRON I’m not going to enumerate the many shortcomings of wp\_cron. But, what’s really cool about WP\_Queue is I can set up my own system to run it. For optimal perfomance, I don’t want to rely on WordPress to trigger wp\_cron and I don’t want to add more “cron” jobs to its queue. Instead, I’d rather add a REST API endpoint and ping it with an external cron job, such as one run with setCronJob.com or something. We can access the queue with the function wp\_queue(). That returns a Queue object that has a public method called worker(). That returns a Worker object with a process() method that will run one job if any jobs are available to be processed. We can use a while loop to run jobs until process returns false, and use an incrementing integer to keep the loop from running for too many requests: View the code on [Gist](https://gist.github.com/Shelob9/1d8756c30330d291009e52a51c59a40e). In this example, I’m hooking it up to a REST API route, that would allow an external cron service to trigger the queue. You manage your server, it would be most secure to trigger that function with a real cron or a command that can only be run from inside of the server/ container. If that’s not possible, at a public/secret key authentication check or something. ### [Josh Pollock](https://torquemag.io/author/joshp/ "All posts by Josh Pollock") ![](https://secure.gravatar.com/avatar/084f04bd04cbc145d29135eb790a8abf?s=70&d=mm&r=g) [](https://CalderaForms.com)[](https://www.facebook.com/CalderaWP/)[](http://twitter.com/Josh412)[](https://www.youtube.com/channel/UC9HykN0jpm2mPwndnAtwm5Q) [Josh](https://JoshPress.net "Josh's Website") is a WordPress developer and educator. He is the founder of [CalderaWP](http://calderalabs.org), makers of awesome WordPress tools including [Caldera Forms](https://calderaforms.com) — a drag and drop, responsive WordPress form builder. The post [Using the WP Queue to Copy REST API Data to Files](https://torquemag.io/2019/01/using-the-wp-queue-to-copy-rest-api-data-to-files/) appeared first on [Torque](https://torquemag.io).