Tag: api

Troubleshooting ‘The provided URL does not represent a valid oEmbed resource.’ with YouTube videos

If you are trying to add a YouTube video using media > remote video in Drupal 8.7 (media is now part of core) and you stumble across this error – which is not discussed anywhere in the core issue queue that I can see…

The provided URL does not represent a valid oEmbed resource.

the fix is to add this line in settings.php:

$settings['http_client_config']['force_ip_resolve'] = 'v4';

Debugging / cause:

– First verify it’s not a private video (unlisted is fine)
– Optionally you can manually test the oEmbed response in your browser (you should get some JSON data back) – the URL needs to be:
https://www.youtube.com/ombed?url=[an encoded full YouTube URL]

It’s as quick/quicker to look in Recent Log Messages in Drupal first, you may be seeing errors like this:

Client error: `GET https://www.youtube.com/oembed?url=https%3A//www.youtube.com/watch%3Fv%3Dyoutubeid` resulted in a `429 Too Many Requests` response:

and also:

Could not retrieve the oEmbed resource.

(which doesn’t tell you anything).

oEmbed requests in Drupal are simple, unauthenticated HTTP GETs – i.e. they don’t use the YouTube API.  Nothing particularly wrong about this.  However the trouble is YouTube has started blocking IPv6 blocks en-mass; apparently because it’s too easy for people to keep changing IPv6 addresses (if ISPs provide a large pool of them) and use them for spam.   Therefore all and any requests over IPv6 may simply be blocked, regardless of how few you’ve made (I can confirm this for Linode servers.)

First, verify this is true for your server by manually grabbing the URL at the command line via wget, with the  -6 and -4 switches.

Assuming IPv4 works, you now need to tell Drupal to make Drupal use it.  This is easier than expected.   Drupal uses the Guzzle HTTP client library, specifically as an HTTP client factory – but as this StackExchange answer explains  – part of the default setup automatically merges in values from $settings, i.e. anything you add under $settings[‘http_client_config’].

So make your $settings.php writeable, and configure the force_ip_resolve setting as shown at the beginning of this post.

YouTube queries should then start working immediately.

 

 

 

 

WordPress REST API security risks

I’d recommend any WordPress users install the disable-JSON-API plugin.  This will prevent anonymous access to WP REST API endpoints, such as /wp-json/wp/v2/users – which provides a list of usernames (API docs).

The REST API was introduced in WordPress 4.7 and is (unfortunately) on by default with no option in settings to turn it off – the idea is it will in time be used for AJAX in some of the admin system, not just external requests.   There is a good change if you were running 4.7.0 or 4.7.1 and did not immediately upgrade to the 4.7.2 security release posts may have been defaced.

Also be aware that the WordPress readme.html file no longer displays complete version numbers – e.g. it will show 4.7 rather than 4.7.2.  (a step backward, IMHO).  Though you might not be aware however you can get the version number by clicking on the WordPress icon at the left of the admin bar (and of course, WP-CLI users can do: wp core version)

Update: Yahoo! Weather

Since the old Yahoo feeds stopped working, I’ve switched my code to use https://developer.yahoo.com/weather/

Note, contrary to what the documentation says, you don’t  need any API keys or authentication.  You can just make a standard GET request and get JSON data back.

The developer page mentions a rate limit of 2,000 signed calls a day but again there’s no indication if/how this is being enforced.

The format is slightly different.  First you need to construct a YQL query, however this is well documented.

Some example PHP code of my own (using the FuelPHP framework):

// Build query for correct city
$BASE_URL      = "http://query.yahooapis.com/v1/public/yql";
$yql_query     = sprintf('select * from weather.forecast where woeid in (select woeid from geo.places(1) where woeid=%d)',
    \Fuel\Core\Config::get('default_weather_loc_id.yahoo'));

$yql_query_url = $BASE_URL . "?q=" . urlencode($yql_query) . "&format=json";

// Make call with cURL
$session = curl_init($yql_query_url);
curl_setopt($session, CURLOPT_RETURNTRANSFER, true);
$json = curl_exec($session);

if (curl_errno($session)) {
    \Log::error('Yahoo check: '.curl_error($session));
}

$weather_data = json_decode($json);

if (! property_exists($weather_data, 'query')) {
    return false;
}

return $weather_data->query->results->channel;

Note, regardless how much data you request, the results are contained inside a JSON channel object, which is inside results, which in turn is within the main query object.

How to ping the MailChimp v3 API

In version 2.0 of MailChimp’s API, there was a helper/ping endpoint, which is useful for automated monitoring (e.g. Nagios checks.)

This no longer exists in v3.0, which is quite a bit different.

Just to share the recommendation their API support team gave me, use the API Root resource instead, and assuming you don’t need all the data it returns, just append ?fields=account_name and it will only send that specific field back.

I’d recommend logging the HTTP response code too and checking it’s 200, e.g. in PHP:

$http_status = curl_getinfo( $ch, CURLINFO_HTTP_CODE );

Workaround for broken Yahoo! Weather apps

Update – 27 July 2016 – the alternative URL stopped working in mid-April, however you can get weather data via a simple plain, unsigned YQL GET request. See my example.

As indicated in this Reddit thread as well as numerous blogs this past week, Yahoo! have changed their XML weather feed (which provides get current observations and a forecast for the city of your choice) from an open service that responds to simple HTTP GET requests to a locked-down API using OAuth.

Unfortunately – though entirely predictably, given it’s widespread use – this has broken things all over the place. It doesn’t help that the Yahoo Developer blog hasn’t been updated since July 2013.

Fortunately there’s a quick workaround by changing the URL if you don’t have time (or the skills, or the suitable web-hosting) to setup OAuth.

Old broken URL for London:

http://weather.yahooapis.com/forecastrss?w=44418&u=c

New working URL, no OAuth required:  Now doesn’t work either

http://xml.weather.yahoo.com/forecastrss?w=44418&u=c

Caveat: Clearly there’s no guarantee whether [sic] this will continue to work.

Other options: OpenWeatherMap looks promising.

Opinion: A public weather feed shouldn’t require authentication. I mean, it’s the weather. Caching? Yes. Rate limiting? Perhaps. API keys? No (or at least, not a legacy service with a single endpoint.)

Also, if you’re providing a service for free, great, but it’s probably worth also creating a blog or mailing list for announcements and encouraging everyone to sign up to it.

Further light reading (if a little tangential): Adactio on Digital Preservation (2012)