Ansible troubleshooting tips

Last updated Wed 20 Dec 2017

Setting file/directory permissions

When using the File module, the documentation warnings about missing off the leading zero for mode, but if you’re using setuid or setgid, you also need to surround the octal number with quotes, otherwise you’ll get unpredictable results.  Write it like this:

- name: Create docroot ({{ docroot }})
  file:
    path: "{{ docroot }}"
    state: directory
    owner: www-data
    group: www-data
    # the leading 2 means set group ID
    mode: "2775"
  become: true

Debugging failing Git connections

When using the Git module, if the command hangs without an error, use the --verbosecommand line switch when running the playbook, and set a low timeout for the task using async so it finishes quickly.  It might be you are connecting the wrong way (e.g. via https when you should be using SSH) or that the server’s SSL certificate needs to be approved.

More about async and poll settings

Variable expansion syntax

You have a variable called {{ current_site }}, which matches a key in {{ websites }}.  How do you write that? i.e. what’s the YML/Ansible equivalent of PHP’s {$foo}?  AnswerL use square brackets, not extra {{ }}

 

{{ websites[current_site].git_branch }}

Passwords

Note there’s a difference between --ask-pass (for standard SSH) and --ask-become-pass(for sudo) and also that --ask-sudo-pass is deprecated.

SSH passwords in the user module need to be supplied as a hash.  MySQL DB passwords are plaintext.

Nested loops

Having everything called {{ item }} gets ambiguous/messy and you’ll get a warning.  Use loop_control and loop_var to give one list of with_items another name.  Ansible docs

Deprecation warnings

(These aren’t always very well documented.)

[defaults]hostfile option, The key is misleading as it can also be a list of hosts, a directory or a list of paths . This feature will
be removed in version 2.8.

Fix is to edit ansible.cfg and change the following (ie. hostfile -> inventory):

[default]
#hostfile = ./inventory/common/allhosts
inventory = ./inventory/common/allhosts

Then the error will go away.

See also: https://github.com/geerlingguy/drupal-vm/issues/1553

Find where you ansible.cfg file is

run "ansible --version"

ansible 2.4.2.0
 config file = /Users/foo/.ansible.cfg
 [...]

Note how on a Mac it may begin with a period (.)

AWS (Amazon Web Services) modules

Limitations:

  • note that although there is now an s3_sync module, it currently (Ansible v 2.4.2.0) only supports push mode (i.e. uploading files TO amazon, not downloading from it to a local device).   However, you can easily use the AWS CLI command in either direction – and because it’s a sync command, it’s idempotent.
aws s3 sync [--delete] s3://mybucket/path/ /path/on/local/server/

UK mobile networks factsheet

This is a FAQ I’m gradually assembling for technically minded people interested in specific network features and the basics of how mobile number portability actually works.

last updated Sun 19 November 2017

Activating new SIM on the network (Three UK):

This one is specific to Three because I haven’t done it for any other networks recently, although a common piece of advice seems to be 2 hours.

So what we’re talking about here is the time after inserting a brand new SIM (in this case PAYG) into an unlocked phone, before it changes from ‘No Service’ to ‘3’, i.e. until you get an indication the registration process has started.   We’re not talking about porting a number (more on that later).

On the first SIM I used, this was instant and I didn’t even need to reboot the phone, on the second it took almost an hour despite numerous toggling of airplane mode, checking the reception was OK, power cycling the phone and so on.  The advice I was given was to leave it in and powered on for four hours.

Until that point, dialling the new number will give “This number has not been recognised”, because it’s not actually on the network yet.

Also, I found that requesting a port before the new SIM was registered meant that it was lost/ignored/failed (despite supplying the number of the SIM) and I needed to request the port again.

Other thing worth noting -an iPhone will prompt you to install new carrier settings as soon as a new SIM is inserted in the phone, before it’s actually registered on the network.  It does this over WiFi.  You can force it to check by going to Settings > General > About.

UK mobile number porting (MNP):

Brief facts:

  • in the UK it’s still pretty manual (GUI software provided by Syniverse to handle generation/validation of PACs, and files exchanged daily over SFTP between the various MNOs to update the databases).  They pay each other a few pence to handle the porting and several thousand pounds for access to the system.
  • You can easily have three (or more) parties involved – the Original network operator, the Donor Service provider and the Recipient Service provider  (this is because the number still has to be routed via the MNO that originally owned it).  Also either the donor or recipient could be an  MNVO – a service provider that doesn’t own their own network.
  • Timings: Have a look at the chart (usual warning – could now be out of date) on page 13 of the Setup Guide for an indication of what order everything happens in.  Note how the new operator is meant to activate the number on their network before 11am, the original network operator (who owns that block of numbers) reroutes it between 11am and 3pm, and the old operator deactivates it sometime after 4pm (having received confirmation).
  • 1 day to transfer, but only Monday to Fridays, excluding bank holidays, because of the carefully choreographed sequence described above.
  • Voice and text messages are ported separately, because there are separate interconnects between operators for voice and SMS.
  • As should be apparent, records need to get updated in multiple places,  which is why, for example, you can get problems involving failing incoming calls from certain networks.

Why isn’t there just a single database?

Well there is, for issuing and checking PACs.  But the source and destination network need to enable/disable the number on their network and the original owner needs to ensure it’s routed correctly.

 

Further reading:

MNP Operator Steering Group website (start with the MNP Setup Guide)

Porting numbers to Three UK:

On the day of the transfer, you should get a text at around 8am to the Three SIM with your temporary number saying:

From Three: Your phone number 440123456789 should be transferred to us by end of the day. Just switch your phone off and on to confirm the transfer. Thanks.

Wi-Fi calling:

Three and EE support voice and SMS (i.e. you can send/receive texts when only connected to wifi).
O2 and Vodafone only support voice (also: anecdotal observation, couldn’t ever get O2 Wi-Fi calling to login, so gave up on it – Three worked instantly on same network).

Observations:

WiFi calling does add a slight amount of latency to the audio.
Conversely, call setup time (from dialling a number to first ring) is much, much faster.
Although holding the IPSec connection open does use some energy, but I’ve not seen a noticeable effect on iPhone battery life (tested from iOS 10.3.3 to 11.1.1)
WiFi calling does timeout/reconnect periodically (rather less so since iOS 11, in my experience), but I’m yet to ever lose connection during a call.

iPhone tips:

Check status. As well as Three WiFi Call in the status bar, you can check Voice/SMS support via Settings > General > About – click on Carrier, it will toggle between Three 28.4 (the Carrier settings version) and IMS Status: Voice & SMS

If you’re someone who keeps their phone in airplane mode but with wifi turned on most of the time, to save battery/reduce radiation etc., note that when you power-cycle the phone wifi calling won’t work until the Sim card has been unlocked – i.e. you’ll need to turn airplane mode off temporarily (but you can turn it straight off again, you don’t need to wait for the network connection to be established).

Technical info:

On iPhone, WiFi calling uses UDP ports 500 and 4500.

Visual Voicemail:

This isn’t supported by Three, but is by EE and O2.
List of which networks support which Apple features.

How it works on O2:

Everything, including the audio files, is sent/received over mobile data – i.e. if you have a bad connection it will be slow or you will see a message saying Visual Voicemail is not available. (At that point you can still ring the voicemail number – in o2’s case 901).  You can play back any messages already downloaded even if you have no connection.

You can set your greeting through the phone without having to dial the voicemail number.

Useful Visual Voicemail features:

  • you’re not paying a per minute rate to listen to messages (your mobile data allowance is being used, but this will be cheaper)
  • the push notification tells you who each message is from
  • you can obviously play messages in any order / scrub through the audio and see the duration of each message at a glance
  • ability to share the audio with other apps if you want to permanently save a message.
  • in Beta (US only) – auto transcription support (it downloads the message and the transcription is done on the phone).

Voicemail on Three UK:

Three doesn’t use the voicemail notification icon.  Instead you immediately get a text telling you the total number of messages in the mailbox (this uses the REPLACE_SM feature in SMS to overwrite any previous notification – so you don’t get a string of texts you have to delete).

Roaming:

Be aware that most international roaming agreements are still 2G / 3G only.  If you want 4G, you’ll typically need to buy a local SIM.  Most networks will let you look up roaming charges on a per country basis on their website.

WiFi calling usually doesn’t work abroad (e.g. O2).
Network specific apps (TuGo, ThreeinTouch etc.) may do.

Coverage maps/predictions

In addition to the network maps, Ofcom have a mobile coverage predictor.  I’ll simply observe for me it is wrong for every network (including predicting I won’t be able to get one network at all when I actually get good reception.)

Reasons for choosing the ‘big 4’ rather than an MVNO

  • MVNOs tend to get less QoS priority on their parent networks, e.g. on O2, access is tiered – o2’s PayM customers at the top, followed by o2 PAYG and Tesco Mobile, then Giffgaff.   This is usually apparent in speed tests and occasionally whether you can connect at 4G.
  • Free London underground Wi-Fi access
  • Services like tethering, WiFi calling, Visual Voicemail often aren’t available on an MVNO.

What network do I currently use?

I’ve used many (on both PayM and PAYG), but currently Three PAYG.

Reasons:

  • best local coverage (including 4G and VoLTE) – your priority when choosing a network should always be how good is the coverage in the places I will use it the most? This will usually be the deciding factor in whether you’re happy with you’re current network. (Check if the situation has changed every so often, I used not to be able to get Three at all.)
  • WiFi calling & SMS
  • best call pricing of any network (3p/min voice, 2p/text, 1p/MB data).
  • Roaming at same prices in many countries (on PAYG as well as PayM)

I would prioritise coverage, the least “hassle” and support for phone software features you wish to use (e.g. wi-fi calling) over contract prices.

PAYG or PayM?

This may be a moot point if the cost of your next phone wipes out any savings you have made.

Always try a PAYG SIM first to test coverage.

If you’re getting a contract, look at the 30-day deals, and consider if the peace of mind of not being tied into a much longer contract outweighs the extra you will be paying each month for the same data/minutes/text allocation.

The choice depends on your usage patterns, how much hassle you find topping up a PAYG phone and if any features are denied to PAYG users (in Three’s case, tethering/hotspots).   If you’re paying for addons every single month (which, for all networks, all expire after 30 days) you may be doing it wrong.

My rule of thumb: at 1p/MB data, if you’re using an average of 1GB a month (Three’s UK month average usage is 6GB apparently – how are people using that much?) PAYG makes more sense.

Note also, all the main networks currently add an annual increase to contract prices in line with inflation (RPI) – PAYG users avoid this.

Drush Symfony error on Drupal 8.4.x

If you’re working on Drupal core or a development site and get this error when running Drush commands, e.g. drush cr or drush pm-list

PHP Fatal error:  Declaration of Drush\Command\DrushInputAdapter::hasParameterOption() must be compatible with Symfony\Component\Console\Input\InputInterface::hasParameterOption($values, $onlyParams = false) in /Applications/DevDesktop/tools/vendor/drush/drush/lib/Drush/Command/DrushInputAdapter.php on line 27

It’s because in the next Drupal minor release (8.4.x)  Symfony components have been updated to ~3.2 (in 8.3.x they remain at 2.8.x)

This causes a conflict with Drush – you should upgrade to Drush 9, which you can do via:

composer require drush/drush:9.*

One of several GitHub issues on drush-ops/drush

PHP troubleshooting: edit the phpcs config file to fix missing or moved coding standards

If phpcs isn’t working properly and running phpcs -i (to get a list of installed coding standards) gives you an error message like this:

PHP Fatal error: Uncaught UnexpectedValueException: DirectoryIterator::__construct(/path/to/a/set/of/phpcs/coding/standards/that/no/longer/exists): failed to open dir: No such file or directory in /usr/local/Cellar/php-code-sniffer/2.8.1/CodeSniffer.php:2250

The quickest way to fix it is to search for the CodeSniffer.conf file (MacOS users, try /usr/local/etc/PHP_CodeSniffer/CodeSniffer.conf) and then edit the installed_paths key to remove or rename the problem directory.

Your output should then look something like this:

$ phpcs -i
The installed coding standards are MySource, PEAR, PHPCS, PSR1, PSR2, Squiz, Zend, Drupal, DrupalPractice, WordPress, WordPress-Core, WordPress-Docs, WordPress-Extra and WordPress-VIP

Drupal troubleshooting – upgrade warning when you already have latest version

Scenario: you’ve upgraded Drupal core to the latest version (via composer) but Drupal thinks an earlier/previous version is the most recent, despite running drush updb, clearing the cache, running cron etc.

e.g. you’ve upgraded Drupal core to 8.3.1, but still see a “version 8.3.0 available” warning on /admin/reports/status

Simple fix: go to /admin/reports/updates and click “Check Manually” and you’ll see a progress bar and it will pull in the latest list of packages and the warning will go away.

HowTo: Nginx with HTTP2 support on Debian Jessie

The original problem:

  • SPDY has been replaced by HTTP2, which is better in a number of ways
  • As of June 2016, Chrome has dropped support for SPDY
  • HTTP2 uses ALPN
  • ALPN requires OpenSSL 1.0.2
  • Debian Stable (aka Jessie aka v8) and others OSes only had 1.0.1

What’s changed:

Procedure:

  • Add jessie-backports and jessie-nginx-http2 (ansible playbook)
  • Upgrade openssl them from the correct place: sudo apt-get install -t jessie-backports openssl
  • sudo apt-get install nginx-full (which should pull in various libnginx-mod packages)
  • Change any references in your Nginx config files from spdy to http2
  • Run sudo nginx -t to verify the configuration is valid
  • Start server

Verify HTTP2 is working (Chrome or Opera):

Developer Tools, Network tab, reload page, enable the Protocol column, look for H2, which means HTTP2.

Extra step for LetsEncrypt / Certbot compatibility:

A few days after doing this I got the following error when my weekly cronjob for renewing LetsEncrypt certificates ran:

    build/temp.linux-x86_64-2.7/_openssl.c:415:30: fatal error: openssl/opensslv.h: No such file or directory
     #include <openssl/opensslv.h>

And on running it manually I had this:

The following packages have unmet dependencies:
 libssl-dev : Depends: libssl1.0.0 (= 1.0.1t-1+deb8u6) but 1.0.2k-1~bpo8+1 is to be installed
 Recommends: libssl-doc but it is not going to be installed

The solution was just to pull in libssl-dev from jessie-backports too:

apt-get install -t jessie-backports libssl-dev

Note, in my case, I have a git clone of certbot rather than a packaged version, though it is now available as a backport for Debian Jessie.

Troubleshooting: Where has the PhpStorm PHP 7 Compatibility Inspection gone?

If you’ve read this guide to checking PHP code for PHP 7 compatibility / used the inspection in the past, you may notice that it’s apparently disappeared – you’re supposed to be able to choose Run Inspection by Name… and choose PHP 7 Compatibility from the list.

This is because it’s been merged with the Language Level inspection.

So:

  • Preferences > Languages and Frameworks > PHP
  • set desired PHP Language Level under Development environment
  • Run inspection by name…
  • choose Language Level

Thanks to Jetbrains Support for helping me with that..

(Tested with PhpStorm 2017.1 EAP)