Troubleshooting Composer timeouts (IPv6 issue)

If composer hangs and you eventually see this error:

The "https://getcomposer.org/versions" file could not be downloaded: failed to open stream: Operation timed out

it could be a problem with IPv6 (for example if you’ve lost your IPv6 address for some reason.)

On MacOS you can turn IPv6 on and off via the command line.  This:

networksetup -listallnetworkservices

…will give you a list of network interfaces (e.g. Ethernet, Wi-Fi, any VPNs)

networksetup -getinfo Wi-Fi

…will list all it’s settings; IP address, MAC and IPv6 status.

networksetup -setv6off Wi-Fi

…will turn IPv6 off. (To turn it back on, you’ll typically want -setv6automatic, though there’s also an option to set it manually.)

In my experience turning it off fixed the problem immediately, and composer worked straight away.

Obviously it’s preferable to fix your IPv6 and turn it back on, but this may be inconvenient if you have to reboot router etc. or beyond your control if there’s a problem with your ISP.

Useful page to test your IPv6 connection: http://ipv6test.google.com/

See also this GitHub issue

Tested with MacOS El Capitan 10.11.5

Troubleshooting Laravel ‘Class log does not exist’ error

This is a common problem – the error is in Illuminate/Container/Container.php (line 736 or 741).

Forum thread

Common reason: spaces in environment variable values in .env file

Two other fixes that have worked for me:

  • after an apt-get upgrade on Homestead, installing the php7.0-mbstring package fixed it (Laravel needs four PHP extensions: OpenSSL, PDO, Mbstring and Tokenizer)
  • on another installation, changing syntax from:
    $app->environment('production')

    to:

    'production' === env('APP_ENV')

    in a ternary operator in config/app.php was the solution.

 

Chrome, SPDY, HTTP/2, Nginx, NPN, APLN, OpenSSL and Debian

Mattias Geniar has written this up in some detail, but to summarise:

Nginx 1.10 (the new major, stable version) has replaced SPDY with HTTP/2. But as of this week, Chrome now only supports HTTP/2 using ALPN.  ALPN requires OpenSSL 1.0.2.  But stable Debian (and CentOS, and other flavours of Linux) only have 1.0.1 and, right now (it seems to me)  there’s not a lot of hope Debian will back-port the new version: many other services also use OpenSSL, they’d all need to be checked/updated…

If you’re using DotDeb on Jessie (aka Debian 8), the nginx packages (nginx nginx-common nginx-full etc.) will be held back when running apt-get upgrade.

What should I do?  Updated – SOLUTION AVAILABLE! (2 March 2017) A newer OpenSSL has been backported.  You need to add some extra repositories and use a special command to install it, but that’s it. Full Instructions

Troubleshooting remote debugging with Xdebug in PhpStorm

Updated Fri 7 Dec 2018.

Running Xdebug remotely is very useful – you can add breakpoints, inspect and modify the state of all variables/objects and type PHP code into a live console.  No need to manual add var_dump() statements etc.
It should save you a lot of time.

Unfortunately, while the setup should be straightforward, it’s easy to lose the programming time you save spending ages tweaking your Xdebug and PhpStorm configuration to get it to work in the first place. (In fact, I don’t think it’s worked correctly first time for me once.)

Also, you’ve probably got multiple projects, maybe running on different VMs, or using Docker (scroll down for special instructions for Docker on the Mac).

Drupal users:

There’s an element of “it just works” for Xdebug in the server/PhpStorm configuration when using Acquia Dev Desktop or Drupal VM.

Acquia Dev Desktop:

To enable/disable Xdebug:

edit /Applications/DevDesktop/php7_1/bin/php.ini
Uncomment the zend xdebug extension and turn on remote_enable

zend_extension=xdebug.so
xdebug.remote_enable = 1

Restart Apache – via one of:

  • the DevDesktop UI
  • the UI but with keyboard shortcuts, cmd-2 (stop) then cmd-1 (start)
  • CLI: /Applications/DevDesktop/apache/bin/apachectl -k restart

Drupal VM:

To turn on xdebug, make sure you’ve got:

php_xdebug_default_enable: 1

and:

installed_extras:
[...]
- xdebug

Then run vagrant provision to build/rebuild (this won’t affect your site). Leave PhpStorm set to port 9000.

Everyone else (non Drupal):

On other installations, here’s my growing checklist when I try to use Xdebug and nothing happens:

(The following is based on Xdebug 2.4.0RC4 and PhpStorm 2016.2.1)

Step 1: you should identify whether it’s a complete lack of communication between Xdebug and the IDE, or if PhpStorm is receiving the data the data but is setup wrong.

Close and restart PhpStorm.  Make sure PhpStorm is listening for PHP Debug Connections (bottom of Run menu, or green telephone icon top right), then reload your app in the browser.

What you’re looking for is either:

a) an ‘Incoming connection’ dialog the very first time you open a webpage of your app.
or
b) ‘Debug session was finished without being paused” in the Event Log.

Either of these confirms Xdebug is talking to PhpStorm and you can need not worry about your php.ini entries, the possibility you’re using the wrong port or that you forgot to trigger the bookmarklet – goto section B.

If nothing happened at all, go to section A.

A. Communication failure

  • Ensure the bookmarklet (recommended) or browser addon you’re using to start/stop debugging is activated.  You can use developer tools to verify the cookie is being sent.
  • Is Xdebug installed? (check phpinfo() output)
  • Does the Xdebug section of phpinfo() have this warning: “XDEBUG NOT LOADED AS ZEND EXTENSION”?  You’re meant to use zend_extension=xdebug.so, not extension=xdebug.so – go to the wizard, paste in your phpinfo() contents and it’ll generate a fully customised set of commands to run (< 2-3 mins) to download the xdebug extension and compile it for your system. (I’ve had this problem break one site – further info).
  • Make sure the bookmarklet’s cookie is correct, it should use PHPSTORM for the IDE key.    does the browser IDE key match xdebug.idekey in your php.ini file? (Verify correct config file is being loaded – have you confused configs for multiple versions or CLI/fpm – and correct value read from it by creating a test page with phpinfo() output.)  Correction: specifying xdebug.idekey in php.ini is unnecessary.  Note if you view the phpinfo() output, at the top of the Xdebug section there’s are two rows, listing the version and the IDE key. If xdebug.idekey is unset, XDebug will fall back to the value of $_SERVER[‘USER’], e.g. www-data, but regardless, as soon you activate your bookmarklet to start a debug session, the $_COOKIE[‘XDEBUG_SESSION’] is set to ‘PHPSTORM’ and that’s what’s passed through. (To test, start and stop a session whilst refreshing your phpinfo() page and watch the IDE key change.)
  • php.ini – ensure xdebug.remote_enable is set to 1 (then restart php-fpm, refresh page and keep an eye out for ‘Debug session was finished without being paused’ in the Event Log)
  • php.ini – you may well need xdebug.remote_connect_back = 1 too (again, restart php-fpm) – I’ve had multiple dev sites that require it – XDebug may be ignoring the connection from the debug client because it doesn’t match xdebug.remote_host, turning on remote_connect_back makes it ignore this and accept a connection from anywhere – make sure your dev machine is behind a firewall.
  • Check the ports match – this can get reset by mistake.  You may also need to use 9001 rather than the default 9000 if the latter is also being used by php-fpm on your server.  Also, open the Event Log and look for a message like “Can’t start listening for connections from ‘xdebug’: Port 9000 is busy”. Ports must be set correctly in two places – xdebug.remote_port and PhpStorm’s preferences (Languages & Frameworks > PHP > Debug > Xdebug)
  • You can verify both ends of this, the phpinfo() output will confirm what port Xdebug will send on.  You can verify PhpStorm is actually listening on the correct port with the lsof command (this example from macOS):
    wt$  sudo lsof -i :9001
    Password:
    COMMAND    PID USER   FD   TYPE             DEVICE SIZE/OFF NODE NAME
    phpstorm 80592   wt  330u  IPv4 0xaad56de5076720c3      0t0  TCP *:etlservicemgr (LISTEN)

    If PhpStorm is not listening this will be blank.

B. PhpStorm settings

  • By default Xdebug won’t do anything if no breakpoints were found. Check the Event Log pane – there will be a warning message, e.g. “Debug session was finished without being paused”. Is there one?
  • You may have configured Remote debugging to skip certain paths. In Preferences, check Languages & Frameworks > PHP > Debug > Skipped Paths (there’s no warning in Event Log if a script is skipped.)  Are you trying to verify an AJAX request, when, for example in WordPress, you have previously blocked admin-ajax.php to avoid spawning a new debug session every time autosave is triggered?
  • If PhpStorm is always stopping on the first line of the script but you don’t want it to, note that this option is specified in two separate places – one at the bottom of the run menu (“Break at first line in PHP scripts”) and also in Preferences (Languages & Frameworks > PHP > Debug > External connections)
  • Path mappings can get confusing. The left section is the project files that PhpStorm is looking at.  The column on the right needs to the path on the server – so you have a Vagrant box running on a Mac host, don’t use /Users/<name>/…, use /var/www/…, /usr/share/ etc.  Double check PhpStorm has saved what you’ve entered correctly.
  • If nothing’s happening, make sure all your previous debug sessions have finished. Press the stop button in the debug panel (cmd-F2 on Mac) and try reloading the page again. Also note also the Max. simultaneous connections setting.
  • Have you lost the Debug window, e.g. you minimised it and can’t find it again? Press cmd-5 or hover over the square in the far bottom-left of the screen.
  • Struggling to find the Path mappings dialog? Go to Preferences > Languages & Frameworks > PHP > Servers
  • Common mapping errors: Typically failing to map the document root (e.g. say in WordPress, you’ve mapped wp-admin and nothing else.)

Skipping the Incoming Connection dialog

Note: you might want to set the PHP_IDE_CONFIG environment variable on your server. This tells PhpStorm which path mapping configuration it should use so it doesn’t prompt you.  It needs to be written like this:

serverName=my-server.dev

where my-server.dev must match the name configured in Languages & Frameworks > PHP > Servers – as shown below:

PhpStorm - configuring the server name

PhpStorm’s Debug diagnostic tool

Go to: Run > Web Server Debug Validation (instructions here)

PhpStorm, Xdebug, Docker & Mac

Your Docker containers will have an IP address beginning 172.xxx.xxx.xxx (each container gets it’s own – check what it is)  You might assume that’d be enough to avoid conflicts with your real IP address or 127.0.0.1 when specifying xdebug.remote_host.

Note: on macOS, you need to create a symlink to your IP otherwise PhpStorm won’t be able to hear Xdebug.

  1. Create the alias:
    sudo ifconfig lo0 alias 10.254.254.254
  2. Put this new address (10.254.254.254) in:
    – your /etc/hosts file (an entry for your project’s hostname)
    – xdebug.remote_host (whether that’s in php.ini or an environment variable in docker-composer.yml)
    – PhpStorm in Preferences > Languages & Frameworks > PHP > Debug > DBGp Proxy > host

Restart everything and things should start working.

Related info:

How to ping the MailChimp v3 API

In version 2.0 of MailChimp’s API, there was a helper/ping endpoint, which is useful for automated monitoring (e.g. Nagios checks.)

This no longer exists in v3.0, which is quite a bit different.

Just to share the recommendation their API support team gave me, use the API Root resource instead, and assuming you don’t need all the data it returns, just append ?fields=account_name and it will only send that specific field back.

I’d recommend logging the HTTP response code too and checking it’s 200, e.g. in PHP:

$http_status = curl_getinfo( $ch, CURLINFO_HTTP_CODE );

Recover deleted AVI files from an SDHC card when you’ve emptied the trash, then repair them

Scenario: You’ve accidentally deleted a bunch of AVI files on an SDHC memory card. You’ve also emptied the Trash in OS X, which would ordinarily mean they’re permanently lost.

  • Install testdisk via Homebrew
  • Use mount to identify the disk format (in this case a FAT-32 partition from Keedox Trail camera)
  • Unmount the disk, diskutil unmount /dev/<whatever> (or you can open Disk Utility and use the GUI)
  • sudo diskutil (if you don’t run as root, chances are you won’t see the full list of drives)
  • Tell it to create a log when it asks you
  • The drive you want should be displayed without RO to indicate it’s accessible, select it
  • It’ll ask for partition type – for FAT-32 use Intel
  • Follow the rest of this procedure (you navigate to the correct partition, directory, the deleted files are in red, you press : (colon) to select them, then C to copy, choose a destination folder, etc.

AVI files not loading

I now had AVI files which were correctly named (apart from the initial _) and timestamped, but looked about twice as big as they should be and wouldn’t play due to unrecognisable header info.  They wouldn’t open in OS X Quick Look or VLC, DivFix++ kept giving me “Seek error” and trying to recover them by regenerating with ffmpeg:

ffmpeg -i input.avi -c copy output.avi

…didn’t work either.

The equivalent did work in mencoder though, and it’s very fast:

mencoder -idx input.avi -ovc copy -oac copy -o output.avi

I had a whole bunch of files (_ICT0001.AVI, _ICT0002.AVI etc.) for which you can use ‘find’ and a bash script:

find * -name '_ICT*.AVI' -type f -exec ~/avifix.sh {} \;

avifix.sh: (test it on a single file first)

#!/bin/bash
mencoder -idx $1 -ovc copy -oac copy -o repaired-$1

Let’s Encrypt news – client name change and auto renewals

Updated Wed 14 Sep 2016 (new installation guide URL, clarify name change.)

Let’s Encrypt have updated their getting started page, but the following may help anyone trying to understand the latest changes.

If you’re installing from scratch, use Certbot (see below) and start here.  You’ll get custom instructions for your operating system and web server – the client can now be installed via a package on newer systems.

The name

The client (now at version 0.6.0) from letsencrypt-auto to Certbot – to be precise, the project/ecosystem is still called Let’s Encrypt, while Certbot is the EFF’s certificate deployment client.

You’ll find that certbot-auto (a shell script) is an exact copy of the letsencrypt-auto, so all previous commands will still work.

The git repository has also been renamed – the old one is redirected.

Old: https://github.com/letsencrypt/letsencrypt
New: https://github.com/certbot/certbot

You can update the location of your ‘origin’ remote in .git/config

Renewals

You can now renew all your certificates at once.

# to test
~/letsencrypt/certbot-auto renew --dry-run

# to actually do it
~/letsencrypt/certbot-auto renew

There’s some clever stuff going on here:

  • It uses all your previous settings.
  • It renews any certificates that will expire within 30 days.
  • Afterwards you get a list of which were renewed and which were skipped (“not due for renewal yet”)
  • --dry-run  use staging server, so doesn’t count towards API limits.
  • “renew” is designed for unattended use.
  • Remember you still need to reload apache/nginx afterwards.

Example of a cron.weekly script you could use:

#!/bin/bash
/path/to/letsencrypt/certbot-auto renew
service nginx reload