سیاره دروپال

Drupal.org - aggregated feeds in category Planet Drupal
Subscribe to خوراک سیاره دروپال

This first-ever Decoupled Summit at DrupalCon Nashville was a huge hit. Not only did it sell out but the room was packed to the gills, literally standing room only. Decoupled Drupal is a hot topic these days. The decoupled summit was an opportunity to look at the state of decoupled Drupal, analyze pros and cons of decoupling, and look at decoupling strategies and examples. There is lots of interest in decoupling, but there are still many hard problems to solve, and it isn’t the right solution for every situation. This summit was an opportunity to assess the state of best practices.

The summit was organized by Lullabot's Sally Young and Mediacurrent's Matt Davis, two of the innovators in this space.

What is “decoupled Drupal”? 

First, a quick explanation of what “decoupled Drupal” means, in case you haven’t caught the fever yet. Historically, Drupal is used to deliver all the components of a website, an approach that can be called “traditional,” “monolithic,” or “full stack” Drupal. In this scenario, Drupal provides the mechanism to create and store structured data, includes an editorial interface that allows editors to add and edit content and set configuration, and takes responsibility for creating the front-end markup that users see in their browsers. Drupal does it all.

“Decoupled”, or “headless” Drupal is where a site separates these website functions across multiple web frameworks and environments. That could mean managing data creation and storage in a traditional Drupal installation, but using React and Node.js to create the page markup. It could also mean using a React app as an editorial interface to a traditional Drupal site. 

Drupal tools and activity

Drupal core is enabling this activity through a couple of core initiatives:

Drupal and the Drupal community have numerous tools available to assist in creating a decoupled site:

  • Contenta, a pre-configured decoupled Drupal distribution.
  • Waterwheel, an emerging ecosystem of software development kits (SDKs) built by the Drupal community.
  • JSON API, an API that allows consumers to request exactly the data they need, rather than being limited to pre-configured REST endpoints.
  • GraphQL, another API that allows consumers to request only the data they want while combining multiple round-trip requests into one.

There’s lots of activity in headless CMSes. But the competitors are proprietary. Drupal and WordPress are the only end-to-end open source contenders. The others only open source the SDKs.

Highlights of the summit

The summit included several speakers, a business panel, and some demonstrations of decoupled applications. Participants brought up lots of interesting questions and observations. I jotted down several quotes, but it wasn't always possible to give attribution with such an open discussion, so my apologies in advance. Some general reflections from my notes:

Why decouple?
  • More and more sites are delivering content to multiple consumers, mobile apps, TV, etc. In this situation, the website can become just another consumer of the data.
  • It’s easier to find generalist JavaScript developers than expert Drupal developers. Decoupling is one way to ensure the front-end team doesn't have to know anything about Drupal.
  • If you have large teams, a decoupled site allows you to have a clean separation of duties, so the front and back end can work rapidly in parallel to build the site.
  • A modern JavaScript front-end can be fast—although several participants pointed out that a decoupled site is not automatically faster. You still need to pay attention to performance issues.
  • Content is expensive to create; decoupling is a way to re-use it, not just across platforms, but also from redesign to redesign.
  • You could launch a brand new design without making any changes to the back end, assuming you have a well-designed API (meaning an API that doesn't include any assumptions about what the front end looks like). As one participant said, “One day, React won't be cool anymore, we'll need to be ready for the next big thing.”
What are some of the complications?
  • It often or always costs more to decouple than to build a traditional site. There’s additional infrastructure, the need to create new solutions for things that traditional Drupal already does, and the fact that we’re still as a community figuring out the best practices.
  • If you only need a website, decoupling is a convoluted way to accomplish it. Decoupling makes sense when you are building an API to serve multiple consumers.
  • You don’t have to decouple to support other applications. Drupal can be a full-featured website, and also the source of APIs.
  • Some tasks are particularly tricky in a decoupled environment, like previewing content before publishing it. Although some participants pointed out that in a truly decoupled environment preview makes no sense anyway. “We have a bias that a node is a page, but that’s not true in a decoupled context. There is no concept of a page on a smartphone. Preview is complicated because of that.”
  • Many businesses have page-centric assumptions embedded deep into their content and processes. It might be difficult to shift to a model where editors create content that might be deployed in many different combinations and environments. One participant discussed a client that "used all the decoupled technology at their disposal to build a highly coupled CMS." On the other hand, some clients are pure Drupal top to bottom, but they have a good content model and are effectively already "decoupling" their content from its eventual display.
  • Another quote, “Clients trying to unify multiple properties have a special problem; they have to swallow that there will have to be a unified content model in order to decouple. Otherwise, you're building numerous decoupled systems.”
  • Once you are decoupled, you may not even know who is consuming the APIs or how they're being used. If you make changes, you may break things outside of your website. You need to be aware of the dependency you created by serving an API.
Speakers and Panelists

The following is a list of speakers and panelists. These are people and companies you could talk to if you have more questions about decoupling:

  • Sally Young (Lullabot)
  • Matt Davis (Mediacurrent)
  • Jeff Eaton (Lullabot)
  • Preston So (Acquia)
  • Matt Grill (Acquia)
  • Daniel Wehner (TES)
  • Wes Ruvalcaba (Lullabot)
  • Mateu Aguiló Bosch (Lullabot)
  • Suzi Arnold (Comcast)
  • Jason Oscar (Comcast)
  • Jeremy Dickens (Weather.com)
  • Nichole Davison (Edutopia)
  • Baddy Breidert (1xinternet)
  • Christoph Breidert (1xinternet)
  • Patrick Coffey (Four Kitchens)
  • Greg Amaroso (Softvision)
  • Eric Hestenes(Edutopia)
  • David Hwang (DocuSign)
  • Shellie Hutchens (Mediacurrent)
  • Karen Stevenson (Lullabot)
Summary

It was a worthwhile summit, I learned a lot, and I imagine others did as well. Several people mentioned that Decoupled Drupal Days will be taking place August 17-19, 2018 in New York City (there is a link to last year's event). The organizers say it will be “brutally honest, not a cheerleading session.” And they’re also looking for sponsors. I’d highly recommend marking those days on your calendar if you’re interested in this topic!

18 Apr Varnish for Drupal 8: the need for speed Niels A Drupal Drupal 8

Our team had been using Varnish a long time for our Dropsolid Drupal 7 project, and we thought the time had come to get it working for Drupal 8 as well. That is why our CTO, Nick Veenhof, organized a meetup about Caching & Purging in Drupal 8. Niels van Mourik gave an elaborate presentation about the Purge module and how it works.
I definitely recommend watching the video and the slides on his blog. In this blog post, we’ll elaborate and build on what Niels explained to us that day. 
First, let’s start off with a quick crash course on what Varnish actually is and how it can benefit your website.

 

Varnish 101

“Varnish Cache is a web application accelerator also known as a caching HTTP reverse proxy. You install it in front of any server that speaks HTTP and configure it to cache the contents. Varnish Cache is really, really fast. It typically speeds up delivery with a factor of 300 - 1000x, depending on your architecture.” (Source: Varnish-cache.org)

In layman’s terms, Varnish will serve a webpage from its own internal cache if it has it available. This drastically reduces the number of requests to the webserver where your application is hosted. This will, in turn, free up resources on your webserver, so your web application can handle more complicated tasks and more users.

In short, Varnish will make your web application faster and will allow you to scale it more efficiently.


How we use Varnish and Drupal 7

How did things typically work in D7? Well, you’d put a Varnish server with a Drupal compatible Varnish configuration file (vcl) in front of your Drupal 7 site and it would start caching it right away - depending, of course, on what is in the vcl and the headers your Drupal site sent.
Next step would be to install the Varnish module from Drupal.org. This module’s sole purpose is to invalidate your Varnish cache and it does so using telnet. This also requires the Varnish server to be accessible from the Drupal backend. This isn’t always an ideal scenario, certainly not when multiple sites are being served from the same Varnish.

The biggest issue when using Drupal 7 with the Varnish module is that invalidation of content just isn’t smart enough. For instance, if you would update one news item you’d only want that page and the ones where the news item is visible to be removed from Varnish’s cache. But that isn’t possible. This isn’t the module’s fault at all - it’s simply the way Drupal 7 was built. There are a few alternatives that do make it a little smarter, but these solutions aren’t foolproof either.

Luckily, Drupal 8 is a whole new ballgame!


How we use Varnish with Drupal 8

Drupal 8 in itself is very smart at caching and it revolves around the three following main pillars (explained from a page cache perspective, it will also cache parts of pages):

  • Cache tags: A page will get a list of tags based on the content that is on it. For instance if you have a news overview, all rendered news items will be added as tags to that page. Allowing you to invalidate that cache if one of those news items change. 
  • Cache context: A page can be different based based on variables from the current request. For instance if you have a news overview that filters out the news items based on a query parameter. 
  • Cache max-age: A page can be served from cache for X amount of time. After the time has passed it needs to be built up again. 

You can read more about Drupal 8’s new caching system here.


All about invalidation

Niels van Mourik created a module called Purge. This is a modular external cache invalidation framework. It leverages Drupal’s cache system to provide easy access to the cache data so you only need to focus on the communication with an external service. It already has a lot of third-party integrations available like Acquia Purge, Akamai and Varnish Purge.

We are now adding another one to the list: the Dropsolid Purge module.


Why does Dropsolid Purge do and why do I need it?

The Dropsolid Purge module enables you to invalidate caches in multiple Varnish load balancers. It also lets you cache multiple web applications by the same Varnish server. The module was very heavily inspired by the Acquia purge module and we reused a lot of the initial code, because it has a smart way of handling the invalidation through tags, but we’ll get in to that a little later. The problem with the Acquia purge module is that it is designed to work on Acquia Cloud, because it depends on certain environment variables and the Varnish configuration is proprietary knowledge of Acquia. This means that it isn’t usable on another environments. 

We also experimented with the Varnish purge module, but it lacked support for cache invalidation in case you have multiple sites/multisites cached by a single Varnish server. This is because the module actually doesn’t tell Varnish which site it should invalidate pages for, so it just invalidates pages for all the sites. It also doesn’t have the most efficient way of passing along the invalidation requests. It contains two ways of sending invalidation request to Varnish: one by one or bundled together. The one by one option results in a lot of requests if you know that updating a single node could easily invalidate 30 tags. Using the Bundle purger could get you to reach the limit of your header size, but more on that later.


What's in the bag?

Currently we provide the following features:

  • Support for tag invalidation and everything invalidation,
  • The module will only purge tags for the current site by using the X-Dropsolid-Site header,
  • The current site is defined by the name you set in config and the subsite directory,
  • Support for multiple load balancers,
  • There is also a default vcl in the examples folder that contains the logic for the bans.

It can be used for any environment if you just follow the installation instructions in the readme.


Under the hood Preparing and handling the responses for/by Varnish

By default, the module will add two headers to every response it gives:

  • X-Dropsolid-Site: A unique site identifier as a hash based on config (you provide through settings.php) and site parameters:
    • A site name
    • A site environment
    • A site group
    • The path of your site (e.g. sites/default or sites/somesubsite)
  • X-Dropsolid-Purge-Tags: A hashed version of each cache tag on the current page. (hashed to keep the length low and avoid hitting the maximum size of the header)

When the response reaches Varnish, it will save those headers along with the cache object. This will allow us to target these specific cache objects for invalidation.

In our vcl file we also strip those headers, so they aren’t visible by the end user:

sub vcl_deliver { unset resp.http.X-Dropsolid-Purge-Tags; unset resp.http.X-Dropsolid-Site; }
Invalidation of pages

For Drupal 8 we no longer use telnet to communicate with Varnish, but we use a BAN request instead. This request will get sent from our site, and it will only be accepted when it comes from our site.  We currently do this by validating the IP of the request against a list of IPs that are allowed to do BAN requests. 

As we mentioned earlier, we provide two ways of invalidating cached pages in Varnish:

  • Tag invalidation: We invalidate pages which have the same cache tags as we send in our BAN request to Varnish.
  • Everything invalidation: We invalidate all pages which are from a certain site.

Tag invalidation

Just like the Acquia purge module, we send a BAN request which contains a group of 12 hashed cache tags which then will be compared to what Varnish has saved. We also pass along the unique site identifier so we indicate we only want to invalidate for a specific site.

Our BAN request has the following headers:

  • X-Dropsolid-Purge: Unique site identifier
  • X-Dropsolid-Purge-Tags: 12 hashed tags

When Varnish picks up this request, it will go through the following logic:

sub vcl_recv { # Only allow BAN requests from IP addresses in the 'purge' ACL. if (req.method == "BAN") { # Same ACL check as above: if (!client.ip ~ purge) { return (synth(403, "Not allowed.")); } # Logic for banning based on tags # https://Varnish-cache.org/docs/trunk/reference/vcl.html#vcl-7-ban if (req.http.X-Dropsolid-Purge-Tags) { # Add bans for tags but only for the current site requesting the ban ban("obj.http.X-Dropsolid-Purge-Tags ~ " + req.http.X-Dropsolid-Purge-Tags + " && obj.http.X-Dropsolid-Site == " + req.http.X-Dropsolid-Purge); return (synth(200, "Ban added.")); } }

We check if the request comes from an IP that is whitelisted. We then add bans for every cache object that matches our unique site identifier and matches at least one of the cache tags we sent along. 

You can easily test this by updating a node and seeing that Varnish will be serving you a new version of its page. 
 

Everything invalidation

When the everything invalidation is triggered, a BAN request is sent with the following headers:

  • X-Dropsolid-Purge-All: True
  • X-Dropsolid-Purge: Unique site identifier

And we execute the following logic on Varnish’s side:
 

sub vcl_recv { # Only allow BAN requests from IP addresses in the 'purge' ACL. if (req.method == "BAN") { # Same ACL check as above: if (!client.ip ~ purge) { return (synth(403, "Not allowed.")); } # Logic for banning everything if (req.http.X-Dropsolid-Purge-All) { # Add bans for the whole site ban("obj.http.X-Dropsolid-Site == " + req.http.X-Dropsolid-Purge); return (synth(200, "Ban added.")); } } }

When Varnish receives a BAN request with the X-Dropsolid-Purge-All header, it will ban all cache object that have the same unique site identifier. You can easily test this by executing the following command: drush cache-rebuild-external.

Beware: a normal drush cache-rebuild will not invalidate an external cache like Varnish.


Why this matters

To us, this is yet another step in making our cache smarter, our web applications faster and our servers leaner. If you have any questions about this post, you can always leave a comment in the comment section below or open an issue on drupal.org.

Are you looking for a partner that will help you to speed up your site, without having to switch hosting? The Dropsolid Platform helps you to adjust and streamline your development processes, without the typical vendor lock-in of traditional hosting solutions. At Dropsolid, we also offer dedicated hosting, but we never enforce our own platform. Dropsolid helps you to grow your digital business - from every possible angle!

 

Blog overview     Get in touch

CCBot/2.0 (https://commoncrawl.org/faq/)

آخرین ارسال ها

محتواهای محبوب

درباره ما

Author
اینجا دروپال یعنی همه چیز. در مورد دروپال صحبت میکنیم. ماژول هامون رو به اشتراک میزاریم در مورد قالب دروپال ، فروشگاه دروپال، دروپال فارسی و تاریخ شمسی دروپال صحبت میکنیم و هرچیزی که در مورد طراحی سایت با دروپال میدونیم به هم انتقال میدیم. دروپالیون یک سایت شخصی نیست. ما دست همه کسانی که برای پیشرفت دروپال تلاش میکنند رو میفشاریم و با آغوش باز اونها رو در این سایت میپذیریم.

تماس با ما

با ما تماس بگیرید.

logo-samandehi