Google, like many other companies, has a required working from home (WFH) policy
during the COVID-19 crisis.
It has taken me a bit, but now I have found a decent WFH setup.
An iPad Pro (12.9-inch) (2nd generation):
This is a dented, scratched device from 2017 that I bought for €349.99
(incl. tax, customs, and shipping) from a dealer
in the UK
on eBay
(this is the search deeplink that I used).
An iPad Air 2: This is a private device from 2014.
A Lenovo Smart Clock:
This mostly just serves as a photo frame, and of course as a desk clock.
The Sidecar feature, so I can use my iPad Pro
as a second screen with my MacBook Air.
The coolest about this feature is that I can multitask it away (see next bullet)
without the laptop readjusting the screen arrangement.
The Hangouts Meet app
on my iPad Pro, so my laptop performance stays unaffected when I am on a video call.
A nice side-effect is that the camera of the iPad Pro is in the middle of my two screens,
so no weird "looking over the other person" effect when I am on a call.
The Gmail app
on my iPad Air, so I can always have an eye on my email.
(Honorable mention) The iDisplay app on the iPad Air
with the iDisplay server running on the laptop, so I can use the iPad Air as a third screen.
Unfortunately, since I do not have another free USB C port on my laptop,
it is really laggy over Wi-Fi, but works when I really need maximum screen real estate.
(Out of scope) The Free Sidecar project promises to
enable Sidecar on older iPads like my iPad Air 2,
since apparently Apple simply blocks older devices for no particular reason.
It requires temporarily turning off System Integrity Protection,
which is something I cannot (and do not want to) do on my corporate laptop.
A school desk—This
is a desk we had bought earlier on eBay, placed on two kids' chairs to convert it into a standing desk.
Some shoe boxes to elevate the two main screens to eye height.
I had quite some neck pain during the first couple of days.
It is definitely not perfect, but I am quite happy with it now.
I very much want the crisis to be over, but (with the kids back in school),
I could probably get used to permanently working from home.
The Asynchronous Clipboard API
provides direct access to read and write clipboard data.
Apart from text, since Chrome 76, you can also copy and paste image data with the API.
For more details on this, check out my
article on web.dev.
Here's the gist of how copying an image blob works:
Note that you need to pass an array of ClipboardItems to the navigator.clipboard.write() method,
which implies that you can place more than one item on the clipboard
(but this is not yet implemented in Chrome as of March 2020).
I have to admit, I only used to think of the clipboard as a one-item stack,
so any new item replaces the existing one.
However, for example, Microsoft Office 365's clipboard on Windows 10 supports
up to 24 clipboard items.
The generic code for pasting an image, that is, for reading from the clipboard,
is a little more involved.
Also be advised that reading from the clipboard triggers a
permission prompt
before the read operation can succeed.
Here's the trimmed down
example from my article:
constpaste=async()=>{ try{ const clipboardItems =await navigator.clipboard.read(); for(const clipboardItem of clipboardItems){ for(const type of clipboardItem.types){ returnawait clipboardItem.getType(type); } } }catch(err){ console.error(err.name, err.message); } }
See how I first iterate over all clipboardItems
(reminder, there can be just one in the current implementation),
but then also iterate over all clipboardItem.types of each individual clipboardItem,
only to then just stop at the first type and return whatever blob I encounter there.
So far I haven't really payed much attention to what this enables,
but yesterday, I had a sudden epiphany 🤯.
Before I get into the details of multi-MIME type copying,
let me quickly derail to
server-driven content negotiation, quoting straight from MDN:
In server-driven content negotiation, or proactive content negotiation, the browser
(or any other kind of user-agent) sends several HTTP headers along with the URL.
These headers describe the preferred choice of the user.
The server uses them as hints and an internal algorithm chooses the best content
to serve to the client.
A similar content negotiation mechanism takes place with copying.
You have probably encountered this effect before
when you have copied rich text, like formatted HTML, into a plain text field:
the rich text is automatically converted to plain text.
(💡 Pro tip: to force pasting into a rich text context without formatting,
use Ctrl + Shift + v on Windows,
or Cmd + Shift + v on macOS.)
So back to content negotiation with image copying.
If you copy an SVG image, then open macOS
Preview,
and finally click "File" > "New from Clipboard",
you would probably expect an image to be pasted.
However, if you copy an SVG image and paste it into
Visual Studio Code
or into SVGOMG's "Paste markup" field,
you would probably expect the source code to be pasted.
With multi-MIME type copying, you can achieve exactly that 🎉.
Below is the code of a future-proof copy function and some helper methods
with the following functionality:
For images that are not SVGs, it creates a textual representation
based on the image's alt text attribute.
For SVG images, it creates a textual representation based on the SVG source code.
At present, the Async Clipboard API only works with image/png,
but nevertheless the code tries to put a representation in the image's original MIME type
into the clipboard, apart from a PNG representation.
So in the generic case, for an SVG image, you would end up with three representations:
the source code as text/plain, the SVG image as image/svg+xml, and a PNG render as image/png.
constcopy=async(img)=>{ // This assumes you have marked up images like so: // <img // src="foo.svg" // data-mime-type="image/svg+xml" // alt="Foo"> // // Applying this markup could be automated // (for all applicable MIME types): // // document.querySelectorAll('img[src*=".svg"]') // .forEach((img) => { // img.dataset.mimeType = 'image/svg+xml'; // }); const mimeType = img.dataset.mimeType; // Always create a textual representation based on the // `alt` text, or based on the source code for SVG images. let text =null; if(mimeType ==='image/svg+xml'){ text =awaittoSourceBlob(img); }else{ text =newBlob([img.alt],{type:'text/plain'}) } const clipboardData ={ 'text/plain': text, }; // Always create a PNG representation. clipboardData['image/png']=awaittoPNGBlob(img); // When dealing with a non-PNG image, create a // representation in the MIME type in question. if(mimeType !=='image/png'){ clipboardData[mimeType]=awaittoOriginBlob(img); } try{ await navigator.clipboard.write([ newClipboardItem(clipboardData), ]); }catch(err){ // Currently only `text/plain` and `image/png` are // implemented, so if there is a `NotAllowedError`, // remove the other representation. console.warn(err.name, err.message); if(err.name ==='NotAllowedError'){ const disallowedMimeType = err.message.replace( /^.*?\s(\w+\/[^\s]+).*?$/,'$1') delete clipboardData[disallowedMimeType]; try{ await navigator.clipboard.write([ newClipboardItem(clipboardData), ]); }catch(err){ throw err; } } } // Log what's ultimately on the clipboard. console.log(clipboardData); };
// Draws an image on an offscreen canvas // and converts it to a PNG blob. consttoPNGBlob=async(img)=>{ const canvas =newOffscreenCanvas( img.naturalWidth, img.naturalHeight); const ctx = canvas.getContext('2d'); // This removes transparency. Remove at will. ctx.fillStyle ='#fff'; ctx.fillRect(0,0, canvas.width, canvas.height); ctx.drawImage(img,0,0); returnawait canvas.convertToBlob(); };
// Fetches an image resource and returns // its blob of whatever MIME type. consttoOriginBlob=async(img)=>{ const response =awaitfetch(img.src); returnawait response.blob(); };
// Fetches an SVG image resource and returns // a blob based on the source code. consttoSourceBlob=async(img)=>{ const response =awaitfetch(img.src); const source =await response.text(); returnnewBlob([source],{type:'text/plain'}); };
If you use this copy function (demo below ⤵️) to copy an SVG image,
for example, everyone's favorite
symptoms of coronavirus 🦠 disease diagram,
and paste it in macOS Preview (that does not support SVG) or the "Paste markup" field of
SVGOMG, this is what you get:
You can play with this code in the embedded example below.
Unfortunately you can't play with this code in the embedded example below yet,
since webappsec-feature-policy#322
is still open.
The demo works if you open it directly on Glitch.
Programmatic multi-MIME type copying is a powerful feature.
At present, the Async Clipboard API is still limited,
but raw clipboard access is on the radar of the
🐡 Project Fugu team
that I am a small part of.
The feature is being tracked as crbug/897289.
All that being said, raw clipboard access has its risks, too, as clearly pointed out in the
TAG review.
I do hope use cases like multi-MIME type copying that I have motivated in this blog post
can help create developer enthusiasm so that browser engineers and security experts can make sure
the feature gets implemented and lands in a secure way.