| We know that PHP config files has the limit to upload files to server, you know that uploading large files can be a real pain. You have to find the loaded php.ini file, edit the upload_max_filesize and post_max_size settings, and hope that you never have to change servers and do all of this over again. |
| We will do some trick to upload very very large file to server without manipulate any configuration, does not matter how small the limit is. We actually will send files from browser chunk by chunk. Chunk can be as small as our demand. Suppose if our server accept max 1mb request limit, we will send 1mb per request and many more. |
| This will include two parts (1) is to send files chunk by chunk from browser to server and the other part (2) is save file parts into single file. I will use PHP example at backend to save file into server. |
| So below is Javascript parts with description: |
<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <title>File Upload Chunk</title> </head> <body> <div class="form"> <input type="file" id="file"/> <button type="button" id="upload">Upload</button> </div> <div class="toast"> <div id="toast"></div> </div> <script type="text/javascript"> document.getElementById("upload").addEventListener("click", () => { let files = document.getElementById("file").files; if (files.length === 0) { alert("File required"); } else { showToast("Uploading"); doUploadFileChunk(files[0], 1); } }); function doUploadFileChunk(file, chunk) { let chunkLimit = 1024 * 50; // Limit of file chunk send to server is 50 KB for test purpose // Blob from position let fromSlice = (chunk - 1) * chunkLimit; // Blob to position let nextSlice = fromSlice + chunkLimit - 1; // new FormData() let formData = new FormData(); formData.append("file_number", chunk); formData.append("file_name", file.name); formData.append("file_part", file.slice(fromSlice, nextSlice + 1)); // Processing small part of file console.log(`Sending from ${fromSlice} to ${nextSlice} KB`); // Sending to server let xhr = new XMLHttpRequest(); xhr.open("POST", "upload.php", true); xhr.setRequestHeader("Accept", "application/json"); xhr.onreadystatechange = () => { if (xhr.readyState === 4) { console.log("Status = " + xhr.status); try { let response = JSON.parse(xhr.responseText); console.log(response); if (response.uploaded) { let percent = Math.floor((nextSlice / file.size) * 100); if (nextSlice < file.size) { showToast(`Uploading file - ${percent}% completed`); setTimeout(() => { doUploadFileChunk(file, chunk + 1); }, 200); } else { showToast(`Upload completed, processing started`); } } else { alert("Failed to upload file"); } } catch (e) { console.log("Complete = " + xhr.responseText); console.log(e); alert("Failed to upload file"); } } }; xhr.send(formData); } function showToast(text) { document.getElementById("toast").innerHTML = text; } </script> <style type="text/css"> div.form { padding: 20px; } div.toast { padding: 20px; background-color: green; color: white; font-size: 31px; } </style> </body> </html> |
| And respective PHP file is as below: |
<?php $output = ["uploaded" => true, "file" => $_FILES["file_part"]]; $uploadFile = sprintf("storage/%s", $_POST['file_name']); if ($_POST["file_number"] == 1) { if (file_exists($uploadFile)) { unlink($uploadFile); } } $destination = fopen($uploadFile, "a+"); if (FALSE === $destination) die("Failed to open destination"); $handle = fopen($_FILES["file_part"]["tmp_name"], "rb"); if (FALSE === $handle) die("Failed to open blob"); $BUFFER_SIZE=1*1024*1024; // 1MB, bigger is faster while( !feof($handle) ) { fwrite($destination, fread($handle, $BUFFER_SIZE) ); } fclose($handle); fclose($destination); echo json_encode($output); |
 |
Showing posts with label javascript. Show all posts
Showing posts with label javascript. Show all posts
Sunday, December 17, 2023
Using JavaScript to Upload Large Files in Chunks as Parts and Avoid Server Limits
Tuesday, December 5, 2023
Streaming HTTP response in PHP - turn long-running process into realtime UI
| Streaming is not a new concept, it is a data transfer technique which allows a web server to continuously send data to a client over a single HTTP connection that remains open indefinitely. In streaming response comes in chunk rather than sending them at once. In the traditional HTTP request / response cycle, a response is not transferred to the browser until it is fully prepared which makes users wait. |
| Output buffering allows to have output of PHP stored into an memory (i.e. buffer) instead of immediately transmitted, it is a mechanism in which instead of sending a response immediately we buffer it in memory so that we can send it at once when whole content is ready. |
| Each time using echo we are basically telling PHP to send a response to the browser, but since PHP has output buffering enabled by default that content gets buffered and not sent to the client. But we will tell PHP to send output immediately then appear rather keep them wait until execution completed. |
| Php script will be like below which will usually send content chunk by chunk: |
<?php // Making maximum execution time unlimited set_time_limit(0); // Send content immediately to the browser on every statement that produces output ob_implicit_flush(1); // Deletes the topmost output buffer and outputs all of its contents ob_end_flush(); sleep(1); echo sprintf("data: %s%s", json_encode(["content" => "Stream 1"]), "\n\n"); sleep(2); echo sprintf("data: %s%s", json_encode(["content" => "Stream 2"]), "\n\n"); sleep(3); echo sprintf("data: %s%s", json_encode(["content" => "Stream 3"]), "\n\n"); exit; |
|
Output buffers catch output given by the program. Each new output buffer is placed on the top of a stack of output buffers, and any output it provides will be caught by the buffer below it. The output control functions handle only the topmost buffer, so the topmost buffer must be removed in order to control the buffers below it.
✔ The ob_implicit_flush(1) enables implicit flushing which sends output directly to the browser as soon as it is produced. ✔ If you need more fine grained control then use flush() function. To send data even when buffers are not full and PHP code execution is not finished we can use ob_flush and flush. The flush() function requests the server to send it's currently buffered output to the browser How to get and process the response in javascript There is a simple example how we can do it with traditional xhr ( XMLHTTPRequest ) request |
function doCallXHR() { let lastResponseLength = 0; let xhr = new XMLHttpRequest(); xhr.open("POST", "/do", true); xhr.setRequestHeader("Content-Type", "application/json"); xhr.setRequestHeader("Accept", "application/json"); xhr.onprogress = (e:any) => { let response = e.currentTarget.response; let progressResponse = lastResponseLength > 0 ? response.substring(lastResponseLength) : response; lastResponseLength = response.length; console.log(new Date().toUTCString()); progressResponse.split(/\n\n/g).filter((t) => t.trim().length > 0).forEach((line) => { console.log(JSON.parse(line.substring(6))); }); }; xhr.onreadystatechange = () => { if (xhr.readyState == 4) { console.log("Status = " + xhr.status); console.log("Complete = " + xhr.responseText); } }; xhr.send(); } doCallXHR(); |
Output is as below from browser console:Wed, 10 Jul 2024 13:33:36 GMT {content: 'Stream 1'} Wed, 10 Jul 2024 13:33:38 GMT {content: 'Stream 2'} Wed, 10 Jul 2024 13:33:41 GMT {content: 'Stream 3'} Status=200 Complete=data: {"content":"Stream 1"} data: {"content":"Stream 2"} data: {"content":"Stream 3"} |
|
xhr.onprogress: is the function called periodically with information until the XMLHttpRequest completely finishes
Few points to note ✅ we are sending chunk response from server and in xhr onprogress getting every new response part merged with the previously received part. ✅ it is possible to load the response one at a time as server response is multiple parts & in a format one after another. We can do it by substracting previoud response string length and parsing with JSON.parse What if any error / exception occur! how to react to that? That's easy.. catch the error & respond with a status that the front-end js script can react to |
<?php try { $response = $this->expensiveProcessing(); } catch(\Exception $e) { // Handle the exception echo json_encode([ 'success' => false, 'message' => $e->getCode() . ' - '. $e->getMessage(), 'progress' => 100 ]); ob_end_flush(); exit; } |
|
Configuration for Nginx You need to do few tweaking with nginx server before working with output buffering. fastcgi_buffering off; proxy_buffering off; gzip off; For whichever reason if you don't have access to nginx server configuration then from PHP code you can also achieve the same result via HTTP header header('X-Accel-Buffering: no'); |
Thursday, February 9, 2023
html2pdf.js | Client-side HTML-to-PDF rendering using pure JS JavaScript
Import html2pdf using CDN:
<script src="https://cdnjs.cloudflare.com/ajax/libs/html2pdf.js/0.10.1/html2pdf.bundle.min.js"/> | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Once installed, html2pdf.js is ready to use. The following command will generate a PDF of #element-to-print and prompt the user to save the result:
var element = document.getElementById('element-to-print'); html2pdf(element); | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Below is a advance example using header and footer:
var date = new Date().toISOString().split("T")[0]; var opt = { margin: 0.5, filename: 'work-log-report-' + date + '.pdf', enableLinks: false, pagebreak: { mode: 'avoid-all' }, image: { type: 'jpeg', quality: 0.98 }, html2canvas: { scale: 2 }, jsPDF: { unit: 'in', format: 'a4', orientation: 'portrait' } }; var element = document.getElementById('element-to-print'); html2pdf().from(element).set(opt).toPdf().get('pdf').then(function (pdf) { console.log("Done"); var totalPages = pdf.internal.getNumberOfPages(); for (var i = 1; i <= totalPages; i++) { pdf.setPage(i); pdf.setFontSize(10); pdf.setTextColor(150); pdf.text('Work Log Report - ' + date, (pdf.internal.pageSize.getWidth()/2) - 0.99, 0.35); pdf.text('Page ' + i + ' of ' + totalPages, (pdf.internal.pageSize.getWidth()/2) - 0.3, pdf.internal.pageSize.getHeight() - 0.25); } }).save(); | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Output would be as below:
 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Live Example
html2pdf.js converts any webpage or element into a printable PDF entirely client-side using html2canvas and jsPDF.
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Tuesday, February 7, 2023
Convert URLSearchParams to a JavaScript Object | JavaScript jQuery Get Parameters from URL | Get Params from URL using JavaScript
| Use the Object.fromEntries method and pass the URLSearchParams instance as an argument. This creates a JavaScript object from the parsed query parameters: |
const params = Object.fromEntries( new URLSearchParams(window.location.search) ) // URL: example.com/path?foo=bar&name=futurestudio&num=1 // { foo: 'bar', name: 'futurestudio', num: '1' } |
| Here’s how it works: the URLSearchParams class implements JavaScript’s iterator interface. The iterator interface contains the Iterator#entries method. This entries methods returns an iterator providing an array of [key, value] pairs: |
const params = new URLSearchParams('?foo=bar&name=futurestudio&num=1') const iterator = params.entries() iterator.next() // { done: false, value: ['foo', 'bar'] } iterator.next() // { done: false, value: ['name', 'futurestudio'] } iterator.next() // { done: false, value: ['num', '1] } iterator.next() // { done: true, value: undefined } |
How to Get URL Parameters with JavaScript
| URL parameters (also called query string parameters or URL variables) are used to send small amounts of data from page to page, or from client to server via a URL. They can contain all kinds of useful information, such as search queries, link referrals, product information, user preferences, and more. |
| In modern browsers, this has become a lot easier, thanks to the URLSearchParams interface. This defines a host of utility methods to work with the query string of a URL. |
Assuming that our URL is https://example.com/?product=shirt&color=blue&newuser&size=m, we can grab the query string using window.location.search:
const queryString = window.location.search; console.log(queryString); // ?product=shirt&color=blue&newuser&size=m |
We can then parse the query string’s parameters using URLSearchParams:
const urlParams = new URLSearchParams(queryString); |
|
Then we call any of its methods on the result. For example, URLSearchParams.get() will return the first value associated with the given search parameter: const product = urlParams.get('product') console.log(product); // shirt const color = urlParams.get('color') console.log(color); // blue const newUser = urlParams.get('newuser') console.log(newUser); // empty string |
Use the Object.fromEntries method and pass the URLSearchParams instance as an argument. This creates a JavaScript object from the parsed query parameters:
const params = Object.fromEntries( new URLSearchParams(window.location.search) ) // URL: example.com/path?foo=bar&name=futurestudio&num=1 // { foo: 'bar', name: 'futurestudio', num: '1' } |
|
Checking for the Presence of a Parameter You can use URLSearchParams.has() to check whether a certain parameter exists: console.log(urlParams.has('product')); // true console.log(urlParams.has('paymentmethod')); // false |
|
Getting All of a Parameter’s Values You can use URLSearchParams.getAll() to return all of the values associated with a particular parameter: console.log(urlParams.getAll('size')); // [ 'm' ] //Programmatically add a second size parameter. urlParams.append('size', 'xl'); console.log(urlParams.getAll('size')); // [ 'm', 'xl' ] |
|
Iterating over Parameters URLSearchParams also provides some familiar Object iterator methods, allowing you iterate over its keys, values and entries: const keys = urlParams.keys(), values = urlParams.values(), entries = urlParams.entries(); for (const key of keys) console.log(key); // product, color, newuser, size for (const value of values) console.log(value); // shirt, blue, , m for(const entry of entries) { console.log(`${entry[0]}: ${entry[1]}`); } // product: shirt // color: blue // newuser: // size: m |
Monday, June 17, 2019
JavaScript | JQuery | Check if a variable is type of String | Number | Null | Undefined | Array | Object
console.log(Object.prototype.toString.call("This is string")); console.log(Object.prototype.toString.call(undefined)); console.log(Object.prototype.toString.call(0)); console.log(Object.prototype.toString.call(null)); console.log(Object.prototype.toString.call([])); console.log(Object.prototype.toString.call({})); |
| And respective output is as below: |
[object String] [object Undefined] [object Number] [object Null] [object Array] [object Object] |
Subscribe to:
Posts (Atom)