Troubleshooting Drush with Acquia Dev Desktop

Updated Tue 8 Jan 2019

Symptom: drush updb hangs

If drush updb is hanging, first, run it with the --verbose script to see exactly what’s going on – that will show you the MySQL connections. (It turned out in my case Drush was connecting to the wrong database).

Symptom: drush connects to wrong database

Checklist:

  • Try starting the terminal connection via DevDesktop (the icon top right) and see if that makes a difference
  • Double the database connection details in the appropriate loc_ file in ~/.acquia/DevDesktop/DrupalSettings/
  • Check your drush status output
  • Check what happens if you run drush sqlc
  • Try installing the latest Drush (composer require drush/drush) rather than relying on the version that comes with DevDesktop (at time of writing that’s 8.1.7, whereas Drush is now up to 9.5.2)

My situation: I have two MySQL installations running, the 32-bit DevDesktop supplied version, and a 64-bit mariadb (for a very large site where I was experiencing connection timeouts).  They are on different ports. I had a site where ordinary drush commands and drush sqlc connection went to the correct MySQL server but drush updb used the wrong one (confirmed using the --verbose option – in fact it was connecting to a database with a different name entirely.

I couldn’t figure out what was going on, so I upgraded from Drush 8 to 9 and that fixed it immediately.

Symptom: large Drupal 7 site where cache is extremely slow or fails to clear…

drush cc all

…and eventually PHP runs out of memory (“cannot allocate”).

Check Drush is using the correct version of PHP, it may not be – use drush status or, more simply: drush php

To fix:

$ which drush
/Applications/DevDesktop/tools/drush

$ vim /Applications/DevDesktop/tools/drush

# edit the PHP version: [ -z "$PHP_ID" ] && PHP_ID=php7_1

See also: Acquia Dev Desktop known issues page

Drupal debugging: Invalid argument supplied for foreach() EntityFieldManager.php:490

You might see this error occasionally when installing/uninstall modules, running db updates etc.

The code in EntityFieldManager.php loops through an array of entities – each of which is a bundle, for example the user entity contains all the user fields you may have created.

The trick for debugging is to dump the contents of $bundle_field_map and look for something wrong – e.g. my issue was an empty (NULL) font entity, which had been left behind after attempting to remove the fontyourface module.

Backup the DB (obviously) then look in the key_value table for records where collection = entity.definitions.bundle_field_map – for me it was a case of deleting name = font, value = N; and the problem went away.

(So an alternative to writing code is to look at the DB table first.)

Add a PHP extension to Acquia Dev Desktop (example – memcached)

(Updated 16 May 2020 – after testing with latest DevDesktop 2, built May 13 2020, on macOS High Sierra 10.13.6)

These instructions now refer to PHP 7.3 (which is recommended for Drupal 8, and required for the forthcoming Drupal 9).  There is one extra requirement for 7.3 – the entry pcre.jit = 0 in your php.ini, to avoid a JIT Compilation Failed error message.  (That’s specific to the memcached extension.)

Other than that, you can change all 3 to 2 in all the instructions if you still want 7.2.

Let’s assume you have the latest Acquia Dev Desktop, which comes with PHP (switchable between 7.0, 7.1, 7.2 and 7.3), and you want to install the PHP memcached extension – so you can use Drupal’s memcache module to make your local development a bit faster (and test for any memcached issues on production).

Memcached v3.1.0 added support for PHP 7.3.

But first…


Why still use Dev Desktop in 2020? 

I still use DevDesktop, but only for quickly testing things on a clean Drupal site, and some legacy sites.  Otherwise, nowadays I recommend Laravel Valet (you can use it for loads of frameworks/content management systems besides Laravel. It takes care of almost everything, except setting up your MySQL database.)

I don’t recommend Docker. Running PHP and MySQL directly is blisteringly fast compared to Docker for Mac – usually several seconds every page load, and with ADD you can instantly PHP restarts when changing a config setting (e.g. enabling/disabling Xdebug) compared to having to wait a minute or more for a Lando container to rebuild (Docker containers are immutable; you have to rebuild the container to make a significant PHP.ini change like adding or removing an extension – you can’t just restart the service). Composer is faster (still a bit annoying, of course, but consider what it’s doing in the background). Drush is faster, including rebuilding the cache. Over a day, every day, this all really adds up.  For those of you using laptops, your battery life will be extended too. (Docker can also use quite a bit of CPU when idle).

Running PHP on the host means you’re directly accessing the file system and not having to jump through all the hoops you do with Docker and Vagrant – either slow file accesses, or having to wait for changes to sync.  This is especially true with Drupal given all the caching it does.  

It’s very unpopular to criticise Docker (which I’m not, exactly – more the hype and the rush to use it on a platform it wasn’t designed for, despite the slowness issues many people experience.)  There’s a trade off between performance and configuration – the assumption seems to be you should go for whatever requires the least configuration, but is it really that hard to install the correct version of PHP or MySQL version (how about making a note of what your site runs in your Git repository?)  I’d rather spend a bit longer up front getting the configuration right and have the performance benefit each day after.  Also, if I have a choice of what to learn, I think knowledge of PHP extensions and configuration – given I’m a PHP developer – is more useful than learning loads about containers.

Comments welcome by email. (I use Ansible heavily, by the way, so I’m not opposed to automation.)  


Install steps:

You need to repeat this once for each version of PHP you want the extension available in, e.g. 7.1 or 7.2.  The example below is for 7.3.

Download the extension from here –  https://pecl.php.net/package/memcached/3.1.5 – or you can use the quicker get latest link: pecl.php.net/get/memcached

extract the tar file into its own directory – it’s fine if it’s just in your downloads folder (tar -xvf)

Then, change to that directory and run:

/Applications/DevDesktop/php7_3_x64/bin/phpize

This next line is the important bit – to make it properly compatible with the PHP version DevDesktop is using.  Without this you may get an error like:

Symbol not found: _compiler_globals
Expected in: flat namespace

./configure --with-php-config=/Applications/DevDesktop/php7_3_x64/bin/php-config

Tip: modules that don’t successfully load don’t show up in phpinfo(), so a quick way to see what the error was when PHP attempted to load  them is to do php -v (it’ll show any warnings above the version info) or php -m to get a list of all the extensions loaded and verify yours is OK.

make
make install

Important: if you need to rerun any of these steps, wipe the directory you extracted first and start afresh.

You should see:

Installing shared extensions: /Applications/DevDesktop/php7_3_x64/ext/

There will now be a memcached.so file in that directory. If it hadn’t picked the config details up, it would have written to a different directory, e.g. your default homebrew one.

Add the memcached.so file to your php.ini – you can verify where that is via  Preferences > Config in DevDesktop, but it would normally be:

/Applications/DevDesktop/php7_3_x64/bin/php.ini

extension_dir = "/Applications/DevDesktop/php7_2_x64/ext"

extension = memcached.so
pcre.jit = 0

This doesn’t usually get overwritten when upgrading DevDesktop, but you’ll need to go through this process each time you switch to a newer PHP branch.  It will take you a few minutes at most.

NB: I rather thought PECL had an equivalent install command line switch to load the correct php-config, so you could just run pecl install command to save time, but apparently not? Let me know if I’m wrong.

Useful reading elsewhere:

Howto: Drupal 8 and memcached on Lando

    1. Make sure phpinfo(); (available at /admin/reports/status/php in Drupal for administrators) contains a memcached section – by default Lando will install the memcached PECL package.
    2. Install and enable memcache Drupal module.
    3. configure settings.php:
      $settings['memcache']['servers'] = ['cache:11211' => 'default'];
      $settings['memcache']['bins'] = ['default' => 'default'];
      $settings['memcache']['key_prefix'] = 'mysitename_';
      
      $settings['cache']['default'] = 'cache.backend.memcache';

      Note the hostname for external connections (port 11222) is localhost, but internal connections have a different hostnamecache – to verify, run lando info

      (give it the correct sitename – this is especially important if you ever use memcached for two or more sites on the same server, to avoid conflicts.

      Also, note memcached has no security so if an application knows or can guess another applications’ prefix it can read all the data.

    4. Truncate all MySQL tables beginning with cache (Drupal bootstraps from the database by default and otherwise may continue to use them)
    5. Run drush cr
    6. You can should now work.

Verify connection and performance (e.g. hits/misses and memory used) at /admin/reports/memcache

Way to test an internal connection from PHP in a container, if you can’t install what you need:

php -a
> $socket = fsockopen('cache', "11211", $errno, $errstr);

If you don’t see any errors, it has connected.

Troubleshooting PhpStorm – entire composer.json file shows as invalid

If you’re running the latest PhpStorm (2018.2.2 at time of writing) you might have a composer.json file that’s turned entirely yellow recently, even though it’s valid.

This happens when your composer.json has a “description” key and is due to this bug.

Workarounds:

  • Disable the inspection on the file
  • or remove the “description” temporarily

Drupal – troubleshooting config synchronisation broken CSS when updating theme colours/appearance

Symptom: After deploying configuration changes to production, including a change to the theme (e.g. editing colours, colour scheme or logo on /admin/appearance/settings/bartik), you get a mostly white page because the colours are all missing. Yet when you visit that admin URL everything looks correct, and as soon as you click ‘Save Configuration’ the site is normal again.

What’s going on?

This happens if you’re using a standard Drupal .gitignore file, which typically has the following:

# Ignore paths that contain user-generated content.
sites/*/files
sites/*/private

If you export your configuration and look at color.theme.bartik.yml – you’ll find this at the bottom:

stylesheets:
  - 'public://color/bartik-7a1420bf/colors.css'
files:
  - 'public://color/bartik-7a1420bf/logo.svg'
  - 'public://color/bartik-7a1420bf/colors.css'

Every time the colours are updated a new stylesheet is created in /sites/default/files/color/bartik-[hash], but .gitignore is ignoring these files, so even if you perform a configuration sync correctly, production won’t have the CSS file it needs.

The solution is to add this to .gitignore:

sites/*/files/
!sites/*/files/color/
!sites/*/files/color/*

.gitignore syntax for inverting subdirectories is pretty confusing.  You might need to use git add –force If you see references to “double-globs”, that means **, and note it doesn’t work on all platforms (e.g. Mac).

Drupal – troubleshooting missing blocks

It’s possible to hide a block from the entire site by accident, because of some counter-intuitive behaviour in the block Visibility settings.

By default, when placing a block there are settings for Content Types, Pages and Roles.  Usually if you leave these blank (i.e. don’t select any checkboxes), the relevant tab has a summary label saying ‘No restrictions’ and they are ignored.

The Pages tab behaves a bit differently.  It has Show and Hide radio buttons and a textarea to input URLs.   If you leave the button set to show (the default) and the textarea is blank, the block is visible on all pages (barring any other restrictions such as the content type).

The danger is if you change it to Hide, and specify one or more pages to exclude.  If in future you decide to remove those so the list is empty again, you must set the radio button back to ‘Show’, otherwise it’ll be hidden from all pages, even though the tab will still say ‘No restrictions’.

Specifically the combination in this screenshot hides the block everywhere, as if it were disabled completed:

This block will be hidden, even though it says ‘Not restricted’

So if you can’t figure out why a block isn’t showing, manually click the Pages tab and check the full settings.  (This can be particularly confusing if you’re testing View blocks with complex contextual filters and they’re working in the Views UI preview, but not on the page.)

 

 

Ansible 2.5 change – troubleshooting include_tasks and tags

If you’re upgrading to Ansible 2.5 (released Mar 23), make sure you read this.

Specifically, until now,  if you had tagged an include_tasks item, all the tasks in the file you were including would inherit the tag (or any other attributes). But now:

“attributes applied to an include_* task will not be inherited by the tasks within.”

Symptom: when running a playbook with –tags=bar, you will see a series of lines like this, one for each item in your loop…

included: /Users/wt/Dropbox/ansible/foo/bar.yml for myserver
included: /Users/wt/Dropbox/ansible/foo/bar.yml for myserver
included: /Users/wt/Dropbox/ansible/foo/bar.yml for myserver

…but none of the tasks in bar.yml actually execute.

To fix, either you manually tag everything within (ugly) OR you can use a block (elegant).

Ansible troubleshooting tips

Last updated Wed 20 Dec 2017

Setting file/directory permissions

When using the File module, the documentation warnings about missing off the leading zero for mode, but if you’re using setuid or setgid, you also need to surround the octal number with quotes, otherwise you’ll get unpredictable results.  Write it like this:

- name: Create docroot ({{ docroot }})
  file:
    path: "{{ docroot }}"
    state: directory
    owner: www-data
    group: www-data
    # the leading 2 means set group ID
    mode: "2775"
  become: true

Debugging failing Git connections

When using the Git module, if the command hangs without an error, use the --verbosecommand line switch when running the playbook, and set a low timeout for the task using async so it finishes quickly.  It might be you are connecting the wrong way (e.g. via https when you should be using SSH) or that the server’s SSL certificate needs to be approved.

More about async and poll settings

Variable expansion syntax

You have a variable called {{ current_site }}, which matches a key in {{ websites }}.  How do you write that? i.e. what’s the YML/Ansible equivalent of PHP’s {$foo}?  AnswerL use square brackets, not extra {{ }}

 

{{ websites[current_site].git_branch }}

Passwords

Note there’s a difference between --ask-pass (for standard SSH) and --ask-become-pass(for sudo) and also that --ask-sudo-pass is deprecated.

SSH passwords in the user module need to be supplied as a hash.  MySQL DB passwords are plaintext.

Nested loops

Having everything called {{ item }} gets ambiguous/messy and you’ll get a warning.  Use loop_control and loop_var to give one list of with_items another name.  Ansible docs

Deprecation warnings

(These aren’t always very well documented.)

[defaults]hostfile option, The key is misleading as it can also be a list of hosts, a directory or a list of paths . This feature will
be removed in version 2.8.

Fix is to edit ansible.cfg and change the following (ie. hostfile -> inventory):

[default]
#hostfile = ./inventory/common/allhosts
inventory = ./inventory/common/allhosts

Then the error will go away.

See also: https://github.com/geerlingguy/drupal-vm/issues/1553

Find where you ansible.cfg file is

run "ansible --version"

ansible 2.4.2.0
 config file = /Users/foo/.ansible.cfg
 [...]

Note how on a Mac it may begin with a period (.)

AWS (Amazon Web Services) modules

Limitations:

  • note that although there is now an s3_sync module, it currently (Ansible v 2.4.2.0) only supports push mode (i.e. uploading files TO amazon, not downloading from it to a local device).   However, you can easily use the AWS CLI command in either direction – and because it’s a sync command, it’s idempotent.
aws s3 sync [--delete] s3://mybucket/path/ /path/on/local/server/