Tip for debugging complex MySQL queries if you have PhpStorm

  • Start a new scratch file (Mac: shift-cmd-N)
  • choose ‘MySQL’
  • Paste your query in
  • Reformat code (Mac: opt-cmd-L)

It’ll colour code and tidy everything, including lining up all the field names and aliases, neatly arranging the joins and the where clauses, plus you can hover over opening or closing brackets and see the opposite bracket highlighted.

This is useful when previewing raw Drupal view SQL and trying to understand it.

Troubleshooting ‘The provided URL does not represent a valid oEmbed resource.’ with YouTube videos

If you are trying to add a YouTube video using media > remote video in Drupal 8.7 (media is now part of core) and you stumble across this error – which is not discussed anywhere in the core issue queue that I can see…

The provided URL does not represent a valid oEmbed resource.

the fix is to add this line in settings.php:

$settings['http_client_config']['force_ip_resolve'] = 'v4';

Debugging / cause:

– First verify it’s not a private video (unlisted is fine)
– Optionally you can manually test the oEmbed response in your browser (you should get some JSON data back) – the URL needs to be:
https://www.youtube.com/ombed?url=[an encoded full YouTube URL]

It’s as quick/quicker to look in Recent Log Messages in Drupal first, you may be seeing errors like this:

Client error: `GET https://www.youtube.com/oembed?url=https%3A//www.youtube.com/watch%3Fv%3Dyoutubeid` resulted in a `429 Too Many Requests` response:

and also:

Could not retrieve the oEmbed resource.

(which doesn’t tell you anything).

oEmbed requests in Drupal are simple, unauthenticated HTTP GETs – i.e. they don’t use the YouTube API.  Nothing particularly wrong about this.  However the trouble is YouTube has started blocking IPv6 blocks en-mass; apparently because it’s too easy for people to keep changing IPv6 addresses (if ISPs provide a large pool of them) and use them for spam.   Therefore all and any requests over IPv6 may simply be blocked, regardless of how few you’ve made (I can confirm this for Linode servers.)

First, verify this is true for your server by manually grabbing the URL at the command line via wget, with the  -6 and -4 switches.

Assuming IPv4 works, you now need to tell Drupal to make Drupal use it.  This is easier than expected.   Drupal uses the Guzzle HTTP client library, specifically as an HTTP client factory – but as this StackExchange answer explains  – part of the default setup automatically merges in values from $settings, i.e. anything you add under $settings[‘http_client_config’].

So make your $settings.php writeable, and configure the force_ip_resolve setting as shown at the beginning of this post.

YouTube queries should then start working immediately.

 

 

 

 

PSA – Safari 13.0.1 breaks multiple extensions

Even if you’re still running macOS High Sierrra (I’m on 10.13.6) you’ll receive a Safari update (from 12 to 13).  This will break a number of extensions – including most adblocking – also a vim-based keyboard extension I was using called Vimmy.

So you may want to avoid upgrading, or do as I’ve done and switch your default browser to Firefox etc. for the time being.

macOS Catalina 10.15 – things to be aware/wary of

  • 32-bit apps will no longer be supported at all (currently that still includes Acquia DevDesktop – for running Drupal development sites)
  • the default shell is going to change from Bash to Zsh
  • scripting languages won’t be installed by default (Apple gave Python, Ruby and Perl as examples in a support document, but didn’t specifically mention PHP).  It’ll mean Homebrew will need to be installed differently.   Presumably it won’t be an issue if you’re upgrading a previous macOS version, but, also presumably, it will be if you’re using a new computer, even with Migration Assistant.
  • iTunes has been removed and the Apple Music replacement apparently doesn’t support column view of your music library.
  • John Gruber writes that Catalina is a pain to use due to (a) bugs but also (b) excessive permission alert dialogs.

(As usual, my own policy is not to upgrade to a new macOS release until 9-12 months after it’s come out, to allow everything to settle. I write this – post updated Feb 2020 – on a machine that’s still running High Sierra, which should still get support until late 2020 – although again, Apple have never announced support lifecycle dates.)

Upgrading a Pure Digital Radio to DAB+ – as it happened

  • Notice Global have added a couple of new DAB+ stations on the Digital One multiplex.  Other household radios (including a Roberts Sports DAB 5 – my review) can receive these automatically, but the Pure One Classic (purchased Dec 2009 – £47, Amazon) displays “Upgrade pure.com/upgrade”.  I have been putting this off.
  • Nervously visit Pure website – seem to remember DAB+ upgrade cost £10 or so. Felt sense of irritation at charging for a feature that ought to be free, and that offering it free will boost chance of repeat custom.
  • pure.com/upgrade redirects to http://upgrades.pure.com/uk/update and prompts for serial number.
  • Look at slightly worn barcode (sticky label) on back of radio – consists of six digits, two letters, 4 digits, a space then a 9 – is this it? Maybe the final nine is a check digit. (Later: it is, if you remove the two letters in the middle).
  • Enter serial number on iPhone, rejected.
  • Notice letters are lower case.  Enter again in uppercase in case of sloppy programming.  Still rejected.
  • Probably only supports recent products? Click the “if your product is not listed” link and navigate through Digital Radio/Hi-Fi and One Family. (Each model type has a “Family” suffix – idly wonder how many people will be sufficiently on-brand to understand/care what a “family” is, in this context?)  Click Software, then Software upgrade then another Software upgrade link.  Browser downloads a PDF telling me to go to upgrades.pure.com/uk.
  • Am doing all this on an iPhone in the kitchen where the radio is. Wonder if iOS/Safari or the 1Blocker software I’m using it screwing things up.  Go upstairs and load the site there.  Same thing, just a PDF.
  • Return to upgrades page and read product list properly.  Realise One Classic is listed after all, so I should be using the serial number form after all.
  • Realise I’ve forgotten the serial number.  Go back downstairs with phone.   Re-read the bit about the serial number format:

    “Serial numbers are normally in a 6 number, 2 letter, 6 number format.”

  •  Hmm – mine has 6 numbers, 2 letters, but then a gap and 1 number at the end.  Move radio around in light whilst squinting at barcode.  Is that a scratch in the label, where a number used to be?
  • Wonder what to do.  Can I get the serial number via the radio’s UI? Poke around the settings menu a bit.  Doesn’t look like it.
  • Realise I have a number and a barcode, so if I can just scan the barcode…  With a unhealthy level of cynicism, open iPhone camera app, knowing that it will scan QR codes, but assuming Apple, whilst having invested considerable developer resource in creating an animated poo emoji, have probably deemed that enabling the camera to scan a linear barcode is too niche of a feature to bother adding support.  This assumption is proved correct.
  • Wonder if there’s a website that’ll let me scan a barcode, to save installing an app just for this.  Quickly abandon that idea.
  • Open App Store on phone.  Start typing “barcode scanner”.  Choose “barcode scanner” from autocomplete dropdown rather than barcode scanner free (likely stuffed with ads etc.?) or barcode scanner uk (what’s so country-specific about a barcode scanner app?).
  • Results page:
    1st: “QR Code Reader and Creator” (Ad, 191 reviews, 3.5 stars)
    2nd: “QR Code Reader” (37.4k reviews, 4.5 stars)
    3rd: “BarcodeScanner” (8 reviews, 4 stars)
  • Observe, optimistically, that BarcodeScanner doesn’t mention QR codes at all.  Looks much simpler / duller than rest.  Decide to choose it, based on similarity to own personality.
  • Install BarcodeScanner and open it:

Omschrijving
Met deze barcode scanner kun je een barcode scannen om de inhoud van de betreffende barcode te controleren en zien welk type codering er gebruikt is.

  • (Later determine this is Dutch.)
    Press Scan een barcode link at bottom of screen.
  • After a couple of attempts, scan successful (red moving horizontal line turns green).
  • A number was indeed missing – the ‘1’ had been scratched off.  (To the app’s credit, although I can’t copy paste it, it remembers the last number I scanned when I subsequently reopen it, and it tells me what type of barcode it is.)
  • Also take a photo of the barcode, go back upstairs and enter number for safe keeping in my notes app.
  • Enter correct serial number on website, accepted.  Now asked for current software version.  Back downstairs to check the radio again, do this bit on the iPhone – which doesn’t seem to handle the enabled/disabled state of the Continue button quite right (remains greyed out, but still clickable).
  • Now asked to enter ‘Device Hardware Identifier’.  Wonder how secure this needs to be. Wonder how I’m supposed to get that.  See it involves holding down Menu button then waiting and holding it down a second time.  Presented with a 32-digit hex number on two lines of the screen.  Really? Attempt to enter this in full the first time (rejected), verify I’ve typed it correctly, then re-read the instructions and note they only want the top line.  Note that they could have just trimmed the first 16 characters of the response or limited the maximum length of the text field.  (Grateful I don’t own a different Pure radio, having read account from another user that hardware identifier on Pure One Mini only displayed for two seconds at a time before reverting to radio mode).
  • Next page has a link to DAB+ or DMB-R update
  • Decide at this point things are going to be a lot simpler if I just take the radio upstairs to where the computer is (I’m going to have to connect it to USB anyway).  But first, take a photo of the display, showing the hardware identifier.
  • Find somewhere to plug radio in.  Inevitably involves having to unplugging something else because power adaptor is too big.
  • Enter details on computer again and note so far there’s been nothing about payment (but it hasn’t said the upgrade is free, either).
  • Next page has a heading (which I suspect most people won’t notice) saying “Paid software download” , but also doesn’t have any payment info, just a couple of download links (Windows, Mac).  Maybe it’s free and they forgot to update it.  Also has a bit confirming device hardware identifier and an 8 digit DAB+ unlock key.  Copy and paste this into notes app.  In all honestly, can’t be bothered to fully read the following four bullet points – but do notice one mentions having to turn the tuning dial to enter the code.
  • Download the firmware for Mac.  (Extra point to Pure for providing this, as wasn’t confident a non-Windows version would exist).
  • Take down box of USB cables from shelf and select an appropriate one.
  • Run the installer.  Take time to read the instructions carefully.
  • Set the radio to update mode as described.  Attempt update.  App immediately says radio cannot be found.
  • First suspicion is that it won’t work properly when running via the extra USB socket on my (wired) keyboard, so instead connect to a proper USB port on rear of Mac Mini (predictably, sockets already fully in use, so have to work out which of three HDDs to disconnect.  Requires care as macOS has an external USB connected SSD startup disk.  Trace cables and unplug the Time Machine backup drive instead.  The drive was in power save mode (not spinning) at the time, but that doesn’t stop macOS complaining that I didn’t eject it first.
  • Retry firmware upgrade. (Pure app has a button for this).  This time, takes a lot longer (promising), but fails again (not so much).
  • Decide to repeat but power cycle the radio first. Fails again.
  • Aware that not all USB cables are created equal.  Search in box for another.  Curiously, seems to have a USB symbol on both sides, thus making it more of a pain to insert the correct way around.
  • Radio connected! Update process start, a little bit slow, just over 3 minutes.  To Pure’s credit, a copy of your existing firmware is downloaded first.
  • Radio has restarted, seems OK.  Can receive Radio 3 on BBC multiplex.  Go to the update menu and verify the version number has incremented.
  • Haven’t been asked for an unlock key.  Perhaps I don’t need it after all…
  • Put cables back in box and replace on shelf.  In doing so, knock biscuit tin onto floor.  Retrieve tin (contains actual biscuits) and inspect contents.  Mostly smashed.
  • Try and tune into a DAB+ station.  Opt for Gold.  Appears in the station listing twice, once as Gold on local mux I know I definitely can’t get in this room, and as Gold UK on D1 in DAB+ (40k), which is what I need to test.
  • Tune in.  Display shows “Upgrade – pure.com/upgrade”.  Hmm.
  • Maybe I’ll retune it just to be on the safe side.  Choose Autotune. Wonder as I wait, as I do every time I’ve done this, on the inconsistently of TV/radio tuning menus and whether on not this device has a ‘Purge’ option to get rid of stations or secondary services that no longer exist.
  • Station list looks fine (has recent additions), but get identical upgrade message to before when tuning to DAB+ stations.
  • Go back to the software menu.  Idly try turning the tuning knob and suddenly there’s a DAB+/DMB-R unlock option.  Feel stupid about this. although wonder if, as software has been replaced, it could have been rephrased or prompted me to enter the key.
  • Realise I left the phone with the code in upstairs. Go back to fetch.
  • Correctly guess how to enter it (turning the dial to select the number and A-F – obvious – then pressing the button in to advance to the next one – perhaps not quite so obvious to all).  In hindsight, this was perhaps the most enjoyable stage of the process, as it felt a bit like opening a safe.
  • Message saying code accepted. Check to see if there is a congratulatory exclamation mark at the end.  There isn’t.  Wait and if anything else happens, like the message going away or the radio retuning itself. It doesn’t.
  • Attempt to tune in again. “Station not available”.
  • Power cycle the radio.  Repeat.
  • Try BBC mux.  Fine.  (Always much stronger, even though same mast. Wonder why this is?)
  • Double check aerial fully extended.
  • Try other D1 stations again.  Nothing.
  • Scrolling “intellitext” freezes and radio spontaneously reboots.  Worry about this – is radio too old to cope with new software?
  • Remember (possibly apocryphal?) story of TV installer who, unable to get a picture at customer’s house, proceeded  to dismantle entire television, only to return, defeated, to office and learn of Emley Moor mast collapse.  Decide sensible to check if any transmitter work ongoing.
  • Try and remember what the Digital One website address is.  Vaguely recall how the site looks – reassuring old fashioned – quite narrow horizontally, as designed when smaller desktop displays prevalent – maybe a purple colour? Suspect list of stations is out of date and lists services that don’t exist anymore, like Oneword and Birdsong.  Get mildly depressed at this.
  • Remember the domain ‘getdigitalradio.com’ for some reason.  Enter it.  Ok, this isn’t it, this is that peculiar green one with the animation that looks like it was done in Flash (but isn’t) and the confusing menus/navigation.
  • Note Digital Radio UK copyright statement and link to Radioplayer in footer.  Who are Digital Radio UK? A consortium, but who exactly? Can’t remember.
  • Don’t think this site has any transmitter info, but try the postcode checker anyway.  Note, yet again, the choice of colour scheme is confusing: the heading Good reception has a green background, yet the stations underneath it all have a gradient that’s more yellow than green, which implying to me reception will be weaker.)
  • Google “digital one”.  Top result – ukdigitalradio.com – this is it. Check the news page. No updates since Heart 80s launch and BFBS closure in early 2017.
  • Does have a transmitter list – under coverage.  Including Google map (to their credit, unlike many sites since Google helpfully broke everybody’s maps when they started charging for the Maps API, they’ve made the effort to register a API key).  Doesn’t seem to be anywhere that gives me transmitter status though.
  • Decided to try BBC reception site.  Signal from same mast, after all.  Despite annoyance that they can’t just give me a single page of text, transmitter by transmitter, like on Ceefax, the checker does work properly and they haven’t hidden any technical info – and it shows past faults, including about 10 minutes of downtime for the BBC digital mux on Friday night.  Listed as “fault”. Idly wonder cause of this and what, nowadays, BBC / Arqiva consider a good response time.
  • No current faults though.
  • Move radio around a bit.  BBC reception is stable, but not good as normal.  Back on Digital One, can get one bar of Classic FM, with a lot of gurgling, if I move radio closer to the window.
  • Assume probably due to the weather. (Humid day with thunderstorms.)
  • Decide can’t be bothered to test Gold or other DAB+ stations now and go back upstairs and listen to Simon Mayo on Scala via iTunes.   (I can receive the SDL multiplex directly off air here, but only if I climb a small hill first…)

Possible lessons?

  • Print serial numbers directly onto products if you can (consider wear and tear, impact of cleaning, fading from exposure to daylight – perhaps place on a shielded internal surface, such as the inside of a battery compartment cover).  Barcodes are a useful backup, but use QR-codes for maximum compatibility with smartphones.
  • Use serial numbers codes with a consistent format that’s easy for humans to parse. Use dashes where you have separators.  Maybe copy something familiar, like the 0000-0000-0000-0000 credit/debit card style.
  • Read all the instructions.
  • Assume your users won’t read all the instructions.
  • Make it clear at the start of a process which software updates incur a charge.
  • If you’re asking people to enter more than one code, perhaps you’re doing it wrong. (Can’t a unique hardware identifier give you all the information a serial number can?)
  • Allow for common mistakes in your form validation.
  • Include the instructions on entering any unlock codes again after completing a firmware upgrade.
  • Use straightforward, clear navigation and colour schemes on websites.
  • Test things before upgrading a product – e.g. remember to verify if a particular station/multiplex has good reception that day and in that location.
  • Store biscuits near ground level.
  • Bear all this in mind (including the likelihood of users of all abilities making mistakes) when contemplating marketing material for digital switchover.

Later: reception has improved and I can confirm the DAB+ upgrade has worked.  I’m grateful the radio I bought almost ten years ago will continue to work.

P.S.  Pure have a list showing which of their radios are DAB+ compatible.  It’s worth checking as curiously, it shows that another of their radios, the Tempus 1S, with a rather nicer wooden finish and almost twice the price, and which I bought a year later, isn’t in fact DAB+ compatible, despite having a USB socket.

Troubleshooting empty Laravel Redis queue (cause: APP_NAME change)

Be aware that if you’re using Redis to handle a queue, don’t rename your APP_NAME environment variable while there are jobs on the queue, as they will all mysteriously disappear.

This is because the APP_NAME is used as a prefix to the Redis key, so Laravel will fail to find any jobs that have already been added.

'redis' => [

'client' => env('REDIS_CLIENT', 'predis'),

'options' => [
'cluster' => env('REDIS_CLUSTER', 'predis'),
'prefix' => Str::slug(env('APP_NAME', 'laravel'), '_').'_database_',
],

Source: config/database.php:116

.eu domain names and Brexit – latest news

EU domains are handled by EURid (headquartered in Belgium), on behalf of the EU.  In 2018, the EU said they wanted anyone in the UK who’d registered a .eu domain to hand it back – The Register claimed there were about 300,000 such domains.

EURid have a dedicated Brexit page. You should keep checking this.

Scenario 1 is the ‘leaving without a deal’ scenario (with two months allowed to update your registration details to a postal address in the EU27 or EEA).

Scenario 2 is leaving at the end of the planned transitional period (e.g. Dec 2020)

As of May 2019:

  • all plans are suspended (i.e. UK citizens/companies can register .eu domains as normal), because Article 50 has been extended until 31 October.
  • There’s also a new regulation to allow all EU citizens to register .eu domains regardless of where they are living in the world.  EURid told me this would become effective later in the year, “possibly October”.
  • All this will also apply to Gibraltar (GI).

This means, provided your organisation has an EU citizen, it will be fine to register a domain in the UK.  But you’ll need that person to be the registrant for as long as you renew it.

Unclear yet what proof of citizenship will be required.   I wouldn’t expect it to prevent you from using WHOIS privacy services.

How much do .eu domains cost, anyway?

$15/year to register and renew on Hover.com (they will give you free WHOIS privacy, for individuals only)
£5.99 to register (50% sale price) and £11.99 to renew on 123-reg

(123-reg is slightly more expensive when you convert the currencies)

 

ISO country codes of The Commonwealth members

Greetings, fellow traveller in search of a list of ISO codes for countries in the Commonwealth. Tricky to find one, isn’t it? So to hopefully save you the time, here is one I made for a thing…

There are 53 countries (or were in May 2019) – for the official list see The Commonwealth website.

Below are the ISO 3166-1 alpha 2 codes (that is to say, the two letter codes that most people use, including for domain names).

  // Africa
 'BW','CM','GM','GH','KE','SZ','LS','MW','MU','MZ','NA',
 'NG','RW','SC','SL','ZA','UG','TZ','ZM',
  // Asia
 'BD','BN','IN','MY','PK','SG','LK',
  // Caribbean and Americas
 'AG','BS','BB','BZ','CA','DM','GD','GY','JM','LC','KN','VC','TT',
  // Europe
 'CY','MT','GB',
  // Pacific
 'AU','FJ','KI','NR','NZ','PG','WS','SB','TO','TV','VU'

And here are the countries that competed in the 2018 Commonwealth Games in Australia.  There were 71, but only 68 are listed here, because England, Wales, Scotland and Northern Ireland competed separately, but they all share the same GB code (technically they can identified separately with GB-SCT, GB-NI and GB-WLS):

  // Africa
  'CM','GH','GM','KE','LS','MU','MW','MZ','NA',
  'NG','RW','SC','SH','SL','SZ','TZ','ZA','ZM',
  // Asia
  BD','BN','BW','IN','LK','MY','PK','SG',
  // Caribbean and Americas
  'AG','AI','BB','CA','BM','BS','BV','BZ','DM','GD',
  'GY','FK','JM','KN','KY','LC','MS','TC','TT','VC',
  // Europe
  'CY','GB','GG','GI','IM','JE','MT',
  // Pacific
  'AU','CK','FJ','KI','NF','NR','NZ','NU',
  'PG','SB','TO','TV','UG','VU','WS'

 

Kindle Paperwhite 2018 detailed review

Updated 4 May 2019

This is only after a few day’s ownership – but I’ll keep it updated.  I’m comparing this (a Paperwhite 10th generation) to my previous (refurbished) Kindle basic 7th generation which I used for ~ 2 years.

– The 300dpi screen *is* worth it. Text is sharper at lower sizes.  Individual characters are less ‘damaged’ and faded after screen refreshes or actions like requesting and dismissing the dictionary window etc. Photos and illustrations (still black and white obviously) look great, compared to ‘ok’ on the lesser screen.
– The Paperwhite case is much more comfortable to hold than the harsher, more sharply edged plastic in the previous design.
– Page turns are definitely snappier than the Kindle 7. (In comparison, this has a next generation display, greater RAM and a newer processor).
– It doesn’t crash as often when reading The Times (which I don’t read everyday, but it has crashed once, so there’s clearly still some problem there, my theory being it’s the size of the paper: the .pobi file – the Kindle periodicals format – is ~ 25MB compared to roughly 5MB for the New York Times and much smaller sizes for books.)

Update Feb 2020: it hasn’t crashed in months (software version 5.12.3, Dec 2019)

– Battery life seems a little better (it takes longer to charge, which is always a good sign) – at best my old Kindle was lasting 4-5 days between charges (with heavy use).
– The white front light is better than I expected (very even, works well at night at levels ~7,8,9,10). It’s especially good indoors during daylight where you have an average or low amount of light coming through the window: in these situations it genuinely does boost the background to “paper white”, and helps reduce unwanted shadows from your hands cast on the screen, without you thinking you are looking at an illuminated display.
– The “Invert black and white” option (see Settings, Accessibility) is less good than I expected (the grey background is too prominent).
– The entirely flat screen takes a little getting used to – it’s quite easy to slide your thumb over the next article link at the bottom of newspapers / magazines by accident. However you don’t have the issue of false infra-red triggers by other objects.
– There’s a power save feature on these models.  I believe it’s activated after one hour in standby – the ‘Waking up’ period each time is 3-4 seconds.

When you get it:

– You need to login to Amazon manually
– You need to enter wifi credentials
– It’ll likely not be running the very latest software (mine seemed to update after I’d connected it to a charger)
– You receive a new @kindle.com email address, so you will need to change address books or 3rd-party services such as Instapaper, otherwise documents will continue to go to your old device.

Things you need to do on Amazon site (‘content and devices’ section):

– Change your default Kindle device
– ALSO you have to edit the ‘subscription settings’ for any newspaper or magazine (Change the ‘Deliver future editions to’ setting) – this doesn’t happen for you, even after changing the default device.

Other caveats:

– Vocabulary builder, My Clippings etc. starts from scratch on a new Kindle
– A short USB charging cable I’d been using for my old Kindle, which I chose to plugin through an Apple wired keyboard (the keyboard has 2 USB sockets of it’s own) didn’t work properly with the new Kindle – macOS complained the power drain was too high and kept shutting the ports down. However as soon as I connected the new supplied lead, it worked correctly.

The lesson is to not always trust cables or assume all micro-USB to USB are created equal.

If anyone would like my old (fully working) Kindle or can suggest a good place to recycle in the UK, send me an email.