Web developer.
This post by Juliette Culver is over 12 years old but is still one of my favourite explanations of what’s wrong with Drupal and WordPress for developers.
Nowadays, Statamic (and the Laravel ecosystem in general) is a fantastic alternative if you’re a PHP specialist.
If you are trying to add a YouTube video using media > remote video in Drupal 8.7 (media is now part of core) and you stumble across this error – which is not discussed anywhere in the core issue queue that I can see…
The provided URL does not represent a valid oEmbed resource.
the fix is to add this line in settings.php:
$settings['http_client_config']['force_ip_resolve'] = 'v4';
Debugging / cause:
– First verify it’s not a private video (unlisted is fine)
– Optionally you can manually test the oEmbed response in your browser (you should get some JSON data back) – the URL needs to be:
https://www.youtube.com/ombed?url=[an encoded full YouTube URL]
It’s as quick/quicker to look in Recent Log Messages in Drupal first, you may be seeing errors like this:
Client error: `GET https://www.youtube.com/oembed?url=https%3A//www.youtube.com/watch%3Fv%3Dyoutubeid` resulted in a `429 Too Many Requests` response:
and also:
Could not retrieve the oEmbed resource.
(which doesn’t tell you anything).
oEmbed requests in Drupal are simple, unauthenticated HTTP GETs – i.e. they don’t use the YouTube API. Nothing particularly wrong about this. However the trouble is YouTube has started blocking IPv6 blocks en-mass; apparently because it’s too easy for people to keep changing IPv6 addresses (if ISPs provide a large pool of them) and use them for spam. Therefore all and any requests over IPv6 may simply be blocked, regardless of how few you’ve made (I can confirm this for Linode servers.)
First, verify this is true for your server by manually grabbing the URL at the command line via wget, with the -6 and -4 switches.
Assuming IPv4 works, you now need to tell Drupal to make Drupal use it. This is easier than expected. Drupal uses the Guzzle HTTP client library, specifically as an HTTP client factory – but as this StackExchange answer explains – part of the default setup automatically merges in values from $settings, i.e. anything you add under $settings[‘http_client_config’].
So make your $settings.php writeable, and configure the force_ip_resolve setting as shown at the beginning of this post.
YouTube queries should then start working immediately.
Here, for anyone needing to host a D6 LTS site, is a working Nginx fragment (tested with Nginx 1.10.3).
You can reuse your standard Drupal 8 config for everything else (e.g. images, protecting private files and so on).
# Drupal 6 LTS
index index.php;
location / {
if (!-e $request_filename) {
rewrite ^/(.*)$ /index.php?q=$1 last;
}
}
Ordinarily, for modern Drupal sites, I’d use the following standard try_files statement, but I couldn’t get it serve D6 subpages correctl (it just redirects to the homepage, even with the q=… added – email me if you know why).
# Drupal 8
location / {
try_files $uri /index.php?$query_string;
}
A straightforward problem, but one I’ve wasted time on when setting up a D6 LTS site.
Symptom:
The site is currently not available due to technical problems. Please try again later. Thank you for your understanding.
If you are the maintainer of this site, please check your database settings in the settings.php file and ensure that your hosting provider’s database server is running. For more help, see the handbook, or contact your hosting provider.
The uncommented example line in default.settings.php is:
$db_url = ‘mysql://username:password@localhost/databasename’;
I spent some time verifying usernames/passwords and adjusting ansible scripts, what I hadn’t noticed was I need mysqli (Mysql Improved – which has been around since way with mysql v4.1.3), not mysql.
So remember to check the connection protocol as well as the credentials.
A fairly straightforward problem that won’t be unique to Drupal, but you may run into when migrating PHP applications from other hosts.
I was reviving an old D6 site that had been hosted on another ISP (Hostgator, as it happens) and on setting it up on Acquia DevDesktop (which is a local MAMP stack) found PHP wouldn’t execute as normal.
First, isolate the problem:
<?php echo "Hello, world!"; or <?php phpinfo(); and load itSolution in my case:
# $Id$ # Use PHP56 as default AddHandler application/x-httpd-php56 .php <IfModule mod_suphp.c> suPHP_ConfigPath /opt/php56/lib </IfModule>
Of course any .htaccess files become irrelevant if you move your dev or production sites to Nginx, but it’s a good idea to read through it anyway.
Sandbox modules don’t have a drupal.org/project/foo URL like full contrib modules, and therefore you can’t use composer require drupal/foo to add them.
If you have a Drupal 8 site using drupal-composer/drupal-project, here’s what to edit in composer.json – using a sandbox module of mine as an example.
"drupal-wturrell/ckeditor_remove_elementspath": {
"type": "package",
"package": {
"name": "drupal-wturrell/ckeditor_remove_elementspath",
"version": "0.0.1",
"type": "drupal-module",
"source": {
"url": "https://git.drupal.org/sandbox/wturrell/3018599.git",
"type": "git",
"reference": "8.x-1.x"
}
}
},
2. In require (or run composer require with the name you’ve specified)
"drupal-wturrell/ckeditor_remove_elementspath": "^0.0.1",
You can choose any name you like, but drupal-username/module makes sense to me.
Your sandbox module doesn’t need a composer.json file of it’s own.
Update: Jan 2020 – newer versions of Dev Desktop, such as the following, are fully 64-bit on macOS:
Version: 2 Built: Sep 18 2019 03:15:11 Control panel rev: a0d1c92 Stack rev: bb3d2d8
It’s worth noting the version of MySQL that comes installed with Acquia Dev Desktop 2 (5.6.41-84.1) is still 32-bit (unlike PHP which is now 64-bit). Recently I was testing with a particularly large database and found it kept crashing with things like:
‘MySQL server has gone away’
Lost connection to MySQL server at ‘reading authorization packet’
Intensive operations such as Drupal-to-Drupal migration tended to trigger this.
I used MySQLTuner but the changes made little difference so then it occurred to me the 32-bit version simply wasn’t up to it, so I installed the latest 64-bit mariadb from homebrew instead and moved the relevant databases over to that (I have mariadb running on port 3306 and Dev Desktop’s MySQL on it’s default 33067).
So far it’s been stable.
See also: Acquia Dev Desktop known issues page
Updated Tue 8 Jan 2019
If drush updb is hanging, first, run it with the --verbose script to see exactly what’s going on – that will show you the MySQL connections. (It turned out in my case Drush was connecting to the wrong database).
Checklist:
loc_ file in ~/.acquia/DevDesktop/DrupalSettings/drush status outputdrush sqlccomposer require drush/drush) rather than relying on the version that comes with DevDesktop (at time of writing that’s 8.1.7, whereas Drush is now up to 9.5.2)My situation: I have two MySQL installations running, the 32-bit DevDesktop supplied version, and a 64-bit mariadb (for a very large site where I was experiencing connection timeouts). They are on different ports. I had a site where ordinary drush commands and drush sqlc connection went to the correct MySQL server but drush updb used the wrong one (confirmed using the --verbose option – in fact it was connecting to a database with a different name entirely.
I couldn’t figure out what was going on, so I upgraded from Drush 8 to 9 and that fixed it immediately.
drush cc all
…and eventually PHP runs out of memory (“cannot allocate”).
Check Drush is using the correct version of PHP, it may not be – use drush status or, more simply: drush php
To fix:
$ which drush /Applications/DevDesktop/tools/drush $ vim /Applications/DevDesktop/tools/drush # edit the PHP version: [ -z "$PHP_ID" ] && PHP_ID=php7_1
See also: Acquia Dev Desktop known issues page
You might see this error occasionally when installing/uninstall modules, running db updates etc.
The code in EntityFieldManager.php loops through an array of entities – each of which is a bundle, for example the user entity contains all the user fields you may have created.
The trick for debugging is to dump the contents of $bundle_field_map and look for something wrong – e.g. my issue was an empty (NULL) font entity, which had been left behind after attempting to remove the fontyourface module.
Backup the DB (obviously) then look in the key_value table for records where collection = entity.definitions.bundle_field_map – for me it was a case of deleting name = font, value = N; and the problem went away.
(So an alternative to writing code is to look at the DB table first.)
(Updated 16 May 2020 – after testing with latest DevDesktop 2, built May 13 2020, on macOS High Sierra 10.13.6)
These instructions now refer to PHP 7.3 (which is recommended for Drupal 8, and required for the forthcoming Drupal 9). There is one extra requirement for 7.3 – the entry pcre.jit = 0 in your php.ini, to avoid a JIT Compilation Failed error message. (That’s specific to the memcached extension.)
Other than that, you can change all 3 to 2 in all the instructions if you still want 7.2.
Let’s assume you have the latest Acquia Dev Desktop, which comes with PHP (switchable between 7.0, 7.1, 7.2 and 7.3), and you want to install the PHP memcached extension – so you can use Drupal’s memcache module to make your local development a bit faster (and test for any memcached issues on production).
Memcached v3.1.0 added support for PHP 7.3.
But first…
Why still use Dev Desktop in 2020?
I still use DevDesktop, but only for quickly testing things on a clean Drupal site, and some legacy sites. Otherwise, nowadays I recommend Laravel Valet (you can use it for loads of frameworks/content management systems besides Laravel. It takes care of almost everything, except setting up your MySQL database.)
I don’t recommend Docker. Running PHP and MySQL directly is blisteringly fast compared to Docker for Mac – usually several seconds every page load, and with ADD you can instantly PHP restarts when changing a config setting (e.g. enabling/disabling Xdebug) compared to having to wait a minute or more for a Lando container to rebuild (Docker containers are immutable; you have to rebuild the container to make a significant PHP.ini change like adding or removing an extension – you can’t just restart the service). Composer is faster (still a bit annoying, of course, but consider what it’s doing in the background). Drush is faster, including rebuilding the cache. Over a day, every day, this all really adds up. For those of you using laptops, your battery life will be extended too. (Docker can also use quite a bit of CPU when idle).
Running PHP on the host means you’re directly accessing the file system and not having to jump through all the hoops you do with Docker and Vagrant – either slow file accesses, or having to wait for changes to sync. This is especially true with Drupal given all the caching it does.
It’s very unpopular to criticise Docker (which I’m not, exactly – more the hype and the rush to use it on a platform it wasn’t designed for, despite the slowness issues many people experience.) There’s a trade off between performance and configuration – the assumption seems to be you should go for whatever requires the least configuration, but is it really that hard to install the correct version of PHP or MySQL version (how about making a note of what your site runs in your Git repository?) I’d rather spend a bit longer up front getting the configuration right and have the performance benefit each day after. Also, if I have a choice of what to learn, I think knowledge of PHP extensions and configuration – given I’m a PHP developer – is more useful than learning loads about containers.
Comments welcome by email. (I use Ansible heavily, by the way, so I’m not opposed to automation.)
Install steps:
You need to repeat this once for each version of PHP you want the extension available in, e.g. 7.1 or 7.2. The example below is for 7.3.
Download the extension from here – https://pecl.php.net/package/memcached/3.1.5 – or you can use the quicker get latest link: pecl.php.net/get/memcached
extract the tar file into its own directory – it’s fine if it’s just in your downloads folder (tar -xvf)
Then, change to that directory and run:
/Applications/DevDesktop/php7_3_x64/bin/phpize
This next line is the important bit – to make it properly compatible with the PHP version DevDesktop is using. Without this you may get an error like:
Symbol not found: _compiler_globals
Expected in: flat namespace
./configure --with-php-config=/Applications/DevDesktop/php7_3_x64/bin/php-config
Tip: modules that don’t successfully load don’t show up in phpinfo(), so a quick way to see what the error was when PHP attempted to load them is to do php -v (it’ll show any warnings above the version info) or php -m to get a list of all the extensions loaded and verify yours is OK.
make make install
Important: if you need to rerun any of these steps, wipe the directory you extracted first and start afresh.
You should see:
Installing shared extensions: /Applications/DevDesktop/php7_3_x64/ext/
There will now be a memcached.so file in that directory. If it hadn’t picked the config details up, it would have written to a different directory, e.g. your default homebrew one.
Add the memcached.so file to your php.ini – you can verify where that is via Preferences > Config in DevDesktop, but it would normally be:
/Applications/DevDesktop/php7_3_x64/bin/php.ini
extension_dir = "/Applications/DevDesktop/php7_2_x64/ext" extension = memcached.so pcre.jit = 0
This doesn’t usually get overwritten when upgrading DevDesktop, but you’ll need to go through this process each time you switch to a newer PHP branch. It will take you a few minutes at most.
NB: I rather thought PECL had an equivalent install command line switch to load the correct php-config, so you could just run pecl install command to save time, but apparently not? Let me know if I’m wrong.
Useful reading elsewhere: