Magento performance optimization with Varnish cache

OVERVIEW

After installing Magento on hosting you probably have a desire to optimize its work.

Google offers solutions, which consist on using one of the following options: installing higher-performance hardware, correction for mysql/php configuration, using PHP accelerators, accelerate downloads JS and CSS, the correction assignment expires date for content, customization Magento core files to fit your needs.

Varien released a book titled “Enterprise Edition Whitepaper High Performance eCommerce” which explains the above methods of  Magento optimization. I recommend reading this book and use the described methods of optimization.

I propose another new approach for the optimization of Magento, which was not described in the Internet before. I propose to use Varnish for caching pages.

What is Varnish you can read here. Varnish handles the request, looks in the cache this page and, if such page is found, returns it. If the page is not found in the cache – a request sent to Apache. This approach allows you to create any number of dynamic pages.

EXAMPLE

On the main page you can see poll module. Assume that we have 4 different polls. I propose randomly create 100 different copies of the main page. When user loads main page Varnish gives random copy of the main page. The probability of withdrawal of one of the 4 polls close to 25%.

This caching will give us the static pages to users who simply browse the site and did not do such actions as a vote, adding product to cart, login, etc. Majority of these users, so using this approach can significantly reduce the load on Apache. Once a user has voted, logged, added product to cart, etc. Varnish cache turn off.

INSTALLATION

Download the latest version of Magento and install it.

Create folder app/code/local/Varnish.

Create file app/etc/modules/Varnish.xml :

<?xml version="1.0"?>
<config>
  <modules>
    <Varnish>
      <active>true</active>
      <codePool>local</codePool>
    </Varnish>
  </modules>
</config>

app/code/local/Varnish/controllers/IndexController.php :

<?php
class Varnish_IndexController extends Mage_Core_Controller_Front_Action
{
 public function purgeallAction() { }
}

app/code/local/Varnish/etc/config.xml :

<?xml version="1.0"?>
<config>
    <modules>
       <Varnish>
         <version>0.0.1</version>
       </Varnish>
    </modules>
    <global>
        <models>
            <varnish>
              <class>Varnish_Model</class>
              <resourceModel>varnish_mysql4</resourceModel>
            </varnish>
        </models>
        <events>
            <http_response_send_before>
                <observers>
                    <varnish>
                        <type>singleton</type>
                        <class>varnish/observer</class>
                        <method>varnish</method>
                    </varnish>
                </observers>
            </http_response_send_before>
                <application_clean_cache>
                    <observers>
                        <varnish>
                            <type>singleton</type>
                            <class>varnish/observer</class>
                            <method>purgeAll</method>
                        </varnish>
                    </observers>
                </application_clean_cache>
        </events>
    </global>
    <frontend>
       <routers>
           <varnish>
               <use>standard</use>
               <args>
                   <module>Varnish</module>
                   <frontName>varnish</frontName>
               </args>
           </varnish>
       </routers>
    </frontend>
    <default>
       <varnish>
           <purgeall_key>
               <key>#Fj1nzljh</key>
           </purgeall_key>
       </varnish>
    </default>
</config>

app/code/local/Varnish/Model/Observer.php :

/**
 * Varnish Observer model
 *
 * @category   Varnish
 * @package    Varnish
 */
class Varnish_Model_Observer
{
    private $_request = null;

    public function __construct()
    {

    }

    private function getSecureKey()
    {
        return Mage::getStoreConfig('varnish/purgeall_key/key');
    }

    public function getCookie()
    {
        return Mage::app()->getCookie();
    }

    public function varnish($observer)
    {
        if ($this->isSetNoCacheStable()) {
            return false;
        }

        if ($this->pollVerification()) {
            $this->setNoCacheStable();
            return false;
        }

        if ($this->quoteHasItems()) {
            $this->turnOffVarnishCache();
            return false;
        }

        if ($this->customerIsLogged()) {
            $this->turnOffVarnishCache();
            return false;
        }

        $this->turnOnVarnishCache();
    }

    public function turnOffVarnishCache()
    {
        $this->getCookie()->set('nocache', 1);
    }

    public function turnOnVarnishCache()
    {
        $this->getCookie()->delete('nocache');
    }

    public function quoteHasItems()
    {
        $quote = Mage::getSingleton('checkout/session')->getQuote();
        if ($quote instanceof Mage_Sales_Model_Quote && $quote->hasItems()) {
            return true;
        }
    }

    public function customerIsLogged()
    {
        $customerSession = Mage::getSingleton('customer/session');
        if ($customerSession instanceof Mage_Customer_Model_Session  &&
            $customerSession->isLoggedIn()) {
            return true;
        }
    }

    public function pollVerification()
    {
        $justVotedPollId = (int) Mage::getSingleton('core/session')->getJustVotedPoll();
        if ($justVotedPollId) {
            return true;
        }
    }

    public function setNoCacheStable()
    {
        $this->getCookie()->set('nocache_stable', 1, 0);
    }

    public function isSetNoCacheStable()
    {
        return $this->getCookie()->get('nocache_stable') === 1;
    }

    public function purgeAll()
    {
        try {
            $url = Mage::getBaseUrl().'varnish/index/purgeall/key/'.$this->getSecureKey().'/';
            $ch = curl_init();
            curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'PURGE');
            curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
            curl_setopt($ch, CURLOPT_URL, $url);
            curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
            curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
            $responseBody = curl_exec($ch);
            curl_close ($ch);
        } catch (Exception $e) {
            Mage::log($e->getFile().' '.$e->getLine().' '.$e->getMessage());
        }
    }
}

sudo vim /etc/varnish.vcl :

backend apache {
  .host = "127.0.0.1";
  .port = "8088";
  .max_connections = 30;
}

acl purge {
  "localhost";
}

sub vcl_recv {
  #purge all
  if (req.request == "PURGE") {
    if (!client.ip ~ purge) {
      error 405 "Not allowed.";
    }
    if (req.url ~ "varnish/index/purgeall/key/#Fj1nzljh") {
      purge_hash( ".*" );
    }
  }

  #described below need to create random "DynamicPage" value; this functionallity we use
  #to create several caches for 1 dynamic page.
  #for example, "int i = rand() % 2 + 1;" means that we have 2 different copies of one dynamic page
  C{
   #include
   #include
   char buffer [33];
   int i = rand() % 10 + 1;
   sprintf(buffer, "%d", i);
   VRT_SetHdr(sp, HDR_REQ, "\014DynamicPage:", buffer, vrt_magic_string_end);
  }C

  if (req.request != "GET" &&
  req.request != "HEAD" &&
  req.request != "PUT" &&
  req.request != "POST" &&
  req.request != "TRACE" &&
  req.request != "OPTIONS" &&
  req.request != "DELETE") {
    return (pipe);
  }

  # do not cache POST requests
  if (req.request == "POST") {
    return (pipe);
  }

  #we should not cache any page for Magento backend
  if (req.request == "GET" && (req.url ~ "^/admin") || req.url ~ "^/index.php/admin") {
    return (pass);
  }

  #we should not cache any page for checkout and customer modules
  if (req.request == "GET" && (req.url ~ "^/checkout" || req.url ~ "^/customer")) {
    return (pass);
  }

  #do not cache till session end
  if (req.http.cookie ~ "nocache_stable") {
    return (pass);
  }

  #unique identifier witch tell Varnish use cache or not
  if (req.http.cookie ~ "nocache") {
    return (pass);
  }

  if (req.request == "GET" && (req.url ~ "\.(png|jpg|jpeg|gif)$" || req.url ~ "print.css")) {
    lookup;
  }

  #Even though there are few possible values for Accept-Encoding, Varnish treats
  #them literally rather than semantically, so even a small difference which makes
  #no difference to the backend can reduce cache efficiency by making Varnish cache
  #too many different versions of an object.
  #http://varnish.projects.linpro.no/wiki/FAQ/Compression
  if (req.http.Accept-Encoding) {
    if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
      # No point in compressing these
      remove req.http.Accept-Encoding;
    } elsif (req.http.Accept-Encoding ~ "gzip") {
      set req.http.Accept-Encoding = "gzip";
    } elsif (req.http.Accept-Encoding ~ "deflate") {
      set req.http.Accept-Encoding = "deflate";
    } else {
      # unkown algorithm
      remove req.http.Accept-Encoding;
    }
  }

  return (lookup);
}

sub vcl_pipe {
  # Note that only the first request to the backend will have
  # X-Forwarded-For set.  If you use X-Forwarded-For and want to
  # have it set for all requests, make sure to have:
  # set req.http.connection = "close";
  # here.  It is not set by default as it might break some broken web
  # applications, like IIS with NTLM authentication.
  return (pipe);
}

sub vcl_pass {
  return (pass);
}

sub vcl_hit {
  if (!obj.cacheable) {
    return (pass);
  }
  return (deliver);
}

sub vcl_miss {
  return (fetch);
}

sub vcl_fetch {
  return (deliver);
}

sub vcl_deliver {

  if (obj.hits > 0) {
    set resp.http.X-Cache = "HIT";
  } else {
    set resp.http.X-Cache = "MISS";
  }
}

sub vcl_error {
 set obj.http.Content-Type = "text/html; charset=utf-8";
 synthetic {"
 <?xml version="1.0" encoding="utf-8"?>
 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
 "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
 <html>
 <head>
 <title>"} obj.status " " obj.response {"</title>
 </head>
 <body>
 <h1>Error "} obj.status " " obj.response {"</h1>
 <p>"} obj.response {"</p>
 <h3>Guru Meditation:</h3>
 <p>URL: "} obj.http.bhash {"</p>
 <p>XID: "} req.xid {"</p>
 <hr>
 <address>
 <a href="http://www.varnish-cache.org/">Varnish cache server</a>
 </address>
 </body>
 </html>
 "};
 return (deliver);
}

59 Comments

  • Открытый компьютерный форум |

    Hi there, I found your blog by the use of Google at the same time as looking for a related topic, your web site got here up, it seems good. I have bookmarked to my favourites|added to bookmarks.

  • hong kong wallpaper |

    Howdy, Awesome article. Likely to downside to your website inside internet explorer, may possibly test out this specific? For example still is the market head and also a significant a part of other individuals will leave from the wonderful producing due to this difficulty.

  • Carpet - Carpets - Persian Carpet |

    Thanks for any other excellent post. The place else could anybody get that type of information in such a perfect method of writing? I’ve a presentation next week, and I’m on the search for such info.

  • photonvps coupons |

    You’re in reality a excellent webmaster. The site loading speed is incredible. It seems that you’re doing any unique trick. In addition, The contents are masterpiece. you have performed a excellent task on this topic!

So, what do you think ?