29 July 2011

Get objects and related count in one shot

Sometimes you just need to output the number of objects related to another, but this simple operation can be a major blow performance-wise. I hope this trick I use a lot in my symfonydoctrine developments will save you some time.

Let’s consider a blog that allows you to tag your posts:

BlogPost:
  columns:
    title: string(255)
    body: clob
  relations:
    Tags:
      class: Tag
      foreignAlias: BlogPosts
      refClass: BlogPostTag
      local: blog_post_id
      foreign: tag_id
 
Tag:
  columns:
    name: string(255)
 
BlogPostTag:
  columns:
    blog_post_id:
      type: integer
      primary: true
    tag_id:
      type: integer
      primary: true
  relations:
    BlogPost:
      local: blog_post_id
      foreign: id
      foreignAlias: BlogPostTags
    Tag:
      local: tag_id
      foreign: id
      foreignAlias: BlogPostTags

You can retrieve this schema in the symfony 1.x documentation

Now, we build an admin generator which shows the number of tags per blog post on the list, with 20 results per page. This means we will have 1 SQL request to retrieve the 20 posts and 1 SQL request per post to retrieve the tag count. Taking into account the count request of the pager, we will have a total of 22 requests. This will get worse if we choose to display more blog posts at a time.

There is a way to optimize this with Doctrine!

Add count into the query

Let’s add the calculation of the tag count to the request that retrieves the blog posts.
It could look like that:

  // lib/model/doctrine/BlogPostTable.class.php
  /**
   * Find a blog post by its id.
   * @param integer $id
   * @return BlogPost|false
   */
  public function findById($id)
  {
    // Subquery that counts the number of tags per post.
    $sub_query = '(SELECT COUNT(t.id) FROM BlogPostTag t WHERE blog_post_id = '.$id.')';
 
    $query = $this->createQuery('bp')
      ->select('bp.*')
      ->addSelect($sub_query.' as nb_tags') // the number of tags will be in the nb_tags variable
      ->where('bp.id = ?', $id);
 
    return $query->execute();
  }
Explanations
  • The $subquery counts the number of tags for the blog post in SQL (more about Doctrine 1.2 subqueries).
  • Create a query that retrieves blog post by its id
  • Add the $subquery into the select with an alias ‘nb_tags’. You have to specify what you want to select first to use the addSelect method, otherwise it will not work.
  • Return the execution of the query
Result

The result of the query should be an instance of a Doctrine_Record (false if no blog post is found) which contains the result of the subquery into its protected array $_values. As it is a protected attribute of the Doctrine_Record class it can be accessed in your BlogPost model class.

Create a smart getter

So now that we get the value of ‘nb_tags’ into the hydrated record we can write a getter that returns this value in a smart way.
First of all, you should add an attribute to your model class to store the number of tags:

  // lib/model/doctrine/BlogPost.class.php
 
  /**
   * The number of tags of the blog post.
   * @var Integer
   */
  protected $nb_tags = null;

Then, implement the getNbTags() that will return the value of the ‘nb_tags’ key in the $_values array of the doctrine record. But what if the record has been found by using another query? The ‘nb_tags’ will not exist so you have to test it otherwise you might face an exception. This is how you should write your getter:

  // lib/model/doctrine/BlogPost.class.php
 
  /**
   * Return the number of tags related to the blog post.
   *
   * @return Integer
   */
  public function getNbTags()
  {
    // The number of tags is not yet set
    if (is_null($this->nb_tags)
    {
      // the variable added in the SQL request will be found in the $_values of the doctrine record
      if (isset($this->_values['nb_tags']))
      {
        $this->nb_tags = $this->_values['nb_tags'];
      }
      else
      {
        /**
         * The number of tags has not been set in the SQL request
         * Doctrine will lazy load every tag and count them all.
         * This could be optimized by overwriting the createQuery method,
         * adding a left join to the tag table automatically in BlogPostTable.class.php
         * (beware, it can lead to unwanted side effects)
         */
        $this->nb_tags = $this->getTags()->count();
      }
    }
 
    return $this->nb_tags;
  }

Conclusion

So what have we achieved? Simple: we reduced the number of SQL requests in our admin gen from 22 to 2! One to retrieve the blog posts with the number of related tags, and the other by the Doctrine pager. Obviously, this trick isn’t restricted to admin generators, so think of the many situations where you can use it!

Filed under : Tips, doctrine, symfony — Benjamin Grandfond @ 16 h 08 min

11 April 2011

DevOps conference online!

My conference about adopting DevOps philosophy on Symfony projects is now online! You can see (and listen to) me speaking here: http://symfony.com/video/Paris2011/573

In this presentation you will see what I think is the philosophy behind the DevOps movement and how to start with the 4 important aspects of adopting DevOps:

  • Configuration Management with Puppet
  • Development on the production environment with Vagrant
  • Deployment automation with Fabric
  • Continuous deployment with Jenkins

If you are interested by the DevOps movement and you happen to be in Paris, come to our Paris-DevOps meetups. After the two first meetups hosted by Theodo, the meetup is now traveling to other locations. The next one will be held on the 4th of May at Xebia’s office. More info here: http://parisdevops.fr/

Filed under : Agile and DevOps, Theodo, symfony — Fabrice Bernhard @ 13 h 21 min

2 December 2009

More with symfony: a whole chapter devoted to the sfFacebookConnect Plugin

December just started, and with it its usual christmas spirit, Santa Claus, happy children and…. the symfony advent calendar!

This year the symfony advent calendar is a collection of articles written by different symfony experts:

http://www.symfony-project.org/blog/2009/12/01/one-more-thing

and is already available as a book on Amazon!

http://www.amazon.com/exec/obidos/ASIN/2918390178

I had the chance to contribute and write an article on developing for Facebook with symfony. This was the perfect occasion to finally sit down and write 15 pages on the experience I gathered on this specific subject. I had already collected it in the sfFacebookConnect plugin but it was lacking documentation. Well here it is finally! At least on Amazon and in a few days as part of the new symfony advent calendar.

Enjoy and do not hesitate to make a critical feedback, the article will be included with the plugin and can still evolve a lot!

Filed under : sfFacebookConnectPlugin, symfony — Fabrice Bernhard @ 10 h 42 min

24 August 2009

A Facebook Connect plugin for symfony

As promised during the last symfony live conference, I finally release my current work on a Facebook Connect Plugin for symfony. It is inspired by the good sfFacebookPlugin by Jonathan Todd, which has however been unmaintained for quite some time. Since Facebook’s platform is evolving every week and my focus was not on the Facebook platform but on the Facebook Connect functionality, I decided to create this new plugin.

It is for the moment VERY beta. It is used in two projects, http://www.allomatch.com which is a symfony 1.0/propel project and another project on symfony 1.2/doctrine. It is therefore compatible with both Doctrine and Propel. However some issues remain concerning 1.0 and 1.2 versions regarding some options, the tasks for example.

For the installation, the README is a good start but FAR from complete. I invite you to browse through the code to understand the logic and comment on this post if you have any question regarding installation. This will force me to improve the README.

I intend to improve the documentation in the very near future, so if you are not in a hurry, please wait. However I have already received dozens of mails concerning the current status, so I release it for those who need to start a project using Facebook Connect right now.

Here is the link to the plugin:

http://www.symfony-project.org/plugins/sfFacebookConnectPlugin

And here the presentation made at the sflive conference:

http://www.symfony-live.com/pdf/sflive09fr/theodo-symfony-facebook.pdf

Filed under : sfFacebookConnectPlugin, symfony — Fabrice Bernhard @ 12 h 02 min

3 June 2009

sfEasyGMapPlugin 1.0.4 out

sfEasyGMapPlugin 1.0.4 is out and the good news is : the plugin is the 24th most used symfony plugin among the 457 available on http://www.symfony-project.org/plugins/ ! We are now 5 official developers, not counting all the developers I work with who contribute indirectly.

It all started because I was amazed by the success of the Phoogle library on the Internet despite its limited number of functionalities. And since almost all my projects involved a Google Map I wanted to create a plugin containing all the core functionalities I always reuse. Now I am happy to see the popularity of the plugin and am looking forward further possible developments that will continue in the spirit of including as many core functionalities of Google Maps-based application in the plugin.

New functionalities for the moment include :

  • More precise Mercator projections to convert GPS coordinates into Google Pixel coordinates and back GMapCoord::fromPixToLat, GMapCoord::fromLatToPix, etc.
  • Added the GMapBounds::getBoundsContainingMarkers(…) function
  • Added the GMap::centerAndZoomOnMarkers() function which enables to guess zoom and center of the map to fit the markers. Center is easy to guess. Zoom uses width and height of smallest bound, pixel width and height of the map and Mercator projection
  • Added tomr’s contribution: it is now possible to add multiple controls to the map
  • Added the GMapCoord::distance($coord1, $coord2) function which gives an estimation of the distance between two coordinates
  • Added the very useful function $gMap-> getBoundsFromCenterAndZoom(…) which enables one to calculate server-side the bounds corresponding to specific center coordinates, zoom, and map size. This is the equivalent of the client-side map.setCenter(…,…);map.setZoom(…);map.getBounds(); It uses the Mercator projection formulas as used by the Google Maps
  • A new function $gMapMarker->isInsideBounds($bounds)
  • A lot of unit tests
  • And two new samples

Please, feel free to suggest what you consider typical core functionalities of your Google Maps-based applications.

Filed under : sfEasyGMapPlugin, symfony — Fabrice Bernhard @ 0 h 49 min

13 March 2009

Symfony Google Maps API plugin : sfEasyGMapPlugin v1.0 is out!

sfEasyGmapPlugin is a very easy to use Google Maps API plugin for symfony, inspired by the Phoogle class… but better :-)

A very simple version has been available for a few months but I have now finally released the 1.0 version, with the following new features :

- it is now sf1.2 compatible straight out of the box
- it has some unit tests
- the GMap constructor now takes an array of parameters, which is much more flexible and also more in the symfony coding spirit (Warning : the modification of the GMap constructor should break your application if you used the prior version of sfEasyGMapPlugin)
- there are interesting functions concerning Bounds :
- smallest enclosing bound
- propel criteria “in bounds”
- homothety transformation
- zoomOut transformation
- there are interesting functions concerning conversion from/to lat/lng to/from Google’s pixel coordinates system. These can be very useful if you want to guess the bounds knowing only the center lat/lng, the zoom level and the map’s width/height in pixels. They involve a few mathematical formulas that were not so straightforward, (since you need to understand how Google’s projection works) so trust me, these functions are valuable, even if they only concern power users.

I have also developed a few doctrine-specific functions which are unfortunately not available yet because not generic enough. I will try to release them in the next version.

The official symfony page is here : http://www.symfony-project.org/plugins/sfEasyGMapPlugin

Please feel free to comment on this work in progress !

Filed under : sfEasyGMapPlugin, symfony — Fabrice Bernhard @ 16 h 16 min

6 March 2009

Integrate Wordpress into symfony

What better topic to start this technical blog about symfony than to talk about my experience of integrating Wordpress into symfony !

I was looking for a nice blogging solution for symfony, and all I found was a very simple plugin and a lot of people encouraging me to build my own blog. Even though it is a nice exercise, my philosophy is to not reinvent the wheel. Wordpress is surely the best free blogging tool available, so I preferred to spend time integrating it into my symfony application than to create yet another sfVeryEasyBlogPlugin.

Integrating Wordpress into symfony can be done in three steps : (more…)

Filed under : sfWordpressPlugin, symfony — Fabrice Bernhard @ 16 h 19 min