I need to verify that all the URLs that I have saved in a DB (mysql) work and do not return Error 404 when clicking on them in the different places where they appear on my website. Once verified, all those that do not work will be displayed on the web screen and the place on the web where they are located in order to eliminate them.
For this I have a script in PHP that when the user on the web launches test, it takes about 10 minutes to be able to return the result of all the URLs that do not work, so I would need to be able to execute this process in the background and go saving the results in some other table of the BD.
Right now what I'm doing is getting the URLs from the DB (from 4 different tables) and doing a foreach for each type (I'm attaching a snippet):
$urlsUniversitat = $universitiesModel->getURLUniversities();
$urlsFooter = $adminManagmentModel->getURLSubsection();
$urlsAssignaturesExternes = $acordsModel->getURLsAcords();
$urlAssignaturesUAB = $assignaturesModel->getURLAssignatures();
$failedURLS = array();
//Modificamos el time limit para que no pete
set_time_limit(200000);
$options = array(CURLOPT_RETURNTRANSFER => true, CURLOPT_SSL_VERIFYPEER => false, CURLOPT_FOLLOWLOCATION => true);
//URLS Unis
foreach ($urlsUniversitat as $urlUni){
$urlPrincipal = $urlUni['urlUniversitat'];
$handleURLPrincipal = curl_init($urlPrincipal);
curl_setopt_array($handleURLPrincipal, $options);
$responsePrincipal = curl_exec($handleURLPrincipal);
$urlHeaderPrincipal = curl_getinfo($handleURLPrincipal, CURLINFO_HTTP_CODE);
$urlIntercanvi = $urlUni['urlIntercanvis'];
$handleURLIntercanvi = curl_init($urlIntercanvi);
curl_setopt_array($handleURLIntercanvi, $options);
$responseIntercanvi = curl_exec($handleURLIntercanvi);
$urlHeaderIntercanvi = curl_getinfo($handleURLIntercanvi, CURLINFO_HTTP_CODE);
if ($urlHeaderPrincipal == 404 || $urlHeaderPrincipal == 0){
array_push($failedURLS, array('modul' => "URL Universitat",'redirect' => $urlUni['idUniversitat'], 'url' => $urlPrincipal));
}
if ($urlHeaderIntercanvi == 404 || $urlHeaderIntercanvi == 0){
array_push($failedURLS, array('modul' => "URL Intercanvi", 'redirect' => $urlUni['idUniversitat'], 'url' => $urlIntercanvi));
}
curl_close($handleURLPrincipal);
curl_close($handleURLIntercanvi);
}
//URLS Footer
foreach ($urlsFooter as $url){
$urlPrincipal = $url['urlApartat'];
$handler = curl_init($urlPrincipal);
curl_setopt_array($handler, $options);
$response = curl_exec($handler);
$urlHeader = curl_getinfo($handler, CURLINFO_HTTP_CODE);
if ($urlHeader == 404 || $urlHeader == 0){
array_push($failedURLS, array('modul' => "Footer", 'redirect' => $url['titolApartat'], 'url' => $urlPrincipal));
}
curl_close($handler);
}
The idea would be that when saving the result of the process that runs in the background in the DB, it can then show on the screen the time and day in which the test and the results have been done, as well as being able to run it again.
I'm pretty new to the whole web theme and PHP and I can't think of how I could run this in the background or with what tool to do it, could you give me some advice?
To make the process faster you can lower the curl timeout, so you will receive the 404 error sooner.
Running the process in the batch every day can be a good option.
Simple example to run every day at 12 pm using cron: