Theodo

Conseil et réalisation d'applications webs sur mesure

  

Blog

19 September 2011

“Adopt devops philosophy” at the Open World Forum

Version française plus bas

Save the date! Theodo will be present at the Open World Forum, as I have been selected to talk about “Adpoting devops philosophy” on Friday Sept. 23 at 16:30.

More info about the conference here: http://www.openworldforum.org/Conferences/Adopter-la-philosophie-DevOps

I am very happy to be able to spread the good word in such an important conference! The main purpose will be to explain how devops extends agility and its concepts to the whole lifecycle of an IT project, including deployment and system administration and how this can improve the productivity and responsiveness of your IT organization.

See you there!

–

Réservez votre vendredi 23 septembre ! Theodo sera présent à l’Open World Forum, j’ai en effet été sélectionné pour intervenir comme conférencier sur “Comment adopter la philosophie devops” ce vendredi 23/9 à 16:30.

Plus d’informations sur la conférence ici: http://www.openworldforum.org/Conferences/Adopter-la-philosophie-DevOps

Je suis très content de pouvoir répandre la bonne parole devant un nouveau public. Mon objectif sera d’expliquer comment la philosophie devops étend les concepts d’agilité à tout le cycle de vie d’un projet informatique, déploiement et maintenance incluse et comment ces concepts peuvent augmenter la productivité et la réactivité de votre organisation informatique.

Filed under : Agile and DevOps, Theodo, symfony — Fabrice Bernhard @ 15 h 07 min

29 July 2011

Get objects and related count in one shot

Sometimes you just need to output the number of objects related to another, but this simple operation can be a major blow performance-wise. I hope this trick I use a lot in my symfony + doctrine developments will save you some time.

Let’s consider a blog that allows you to tag your posts:

BlogPost:
  columns:
    title: string(255)
    body: clob
  relations:
    Tags:
      class: Tag
      foreignAlias: BlogPosts
      refClass: BlogPostTag
      local: blog_post_id
      foreign: tag_id
 
Tag:
  columns:
    name: string(255)
 
BlogPostTag:
  columns:
    blog_post_id:
      type: integer
      primary: true
    tag_id:
      type: integer
      primary: true
  relations:
    BlogPost:
      local: blog_post_id
      foreign: id
      foreignAlias: BlogPostTags
    Tag:
      local: tag_id
      foreign: id
      foreignAlias: BlogPostTags

You can retrieve this schema in the symfony 1.x documentation

Now, we build an admin generator which shows the number of tags per blog post on the list, with 20 results per page. This means we will have 1 SQL request to retrieve the 20 posts and 1 SQL request per post to retrieve the tag count. Taking into account the count request of the pager, we will have a total of 22 requests. This will get worse if we choose to display more blog posts at a time.

There is a way to optimize this with Doctrine!

Add count into the query

Let’s add the calculation of the tag count to the request that retrieves the blog posts.
It could look like that:

  // lib/model/doctrine/BlogPostTable.class.php
  /**
   * Find a blog post by its id.
   * @param integer $id
   * @return BlogPost|false
   */
  public function findById($id)
  {
    // Subquery that counts the number of tags per post.
    $sub_query = '(SELECT COUNT(t.id) FROM BlogPostTag t WHERE blog_post_id = '.$id.')';
 
    $query = $this->createQuery('bp')
      ->select('bp.*')
      ->addSelect($sub_query.' as nb_tags') // the number of tags will be in the nb_tags variable
      ->where('bp.id = ?', $id);
 
    return $query->execute();
  }
Explanations
  • The $subquery counts the number of tags for the blog post in SQL (more about Doctrine 1.2 subqueries).
  • Create a query that retrieves blog post by its id
  • Add the $subquery into the select with an alias ‘nb_tags’. You have to specify what you want to select first to use the addSelect method, otherwise it will not work.
  • Return the execution of the query
Result

The result of the query should be an instance of a Doctrine_Record (false if no blog post is found) which contains the result of the subquery into its protected array $_values. As it is a protected attribute of the Doctrine_Record class it can be accessed in your BlogPost model class.

Create a smart getter

So now that we get the value of ‘nb_tags’ into the hydrated record we can write a getter that returns this value in a smart way.
First of all, you should add an attribute to your model class to store the number of tags:

  // lib/model/doctrine/BlogPost.class.php
 
  /**
   * The number of tags of the blog post.
   * @var Integer
   */
  protected $nb_tags = null;

Then, implement the getNbTags() that will return the value of the ‘nb_tags’ key in the $_values array of the doctrine record. But what if the record has been found by using another query? The ‘nb_tags’ will not exist so you have to test it otherwise you might face an exception. This is how you should write your getter:

  // lib/model/doctrine/BlogPost.class.php
 
  /**
   * Return the number of tags related to the blog post.
   *
   * @return Integer
   */
  public function getNbTags()
  {
    // The number of tags is not yet set
    if (is_null($this->nb_tags)
    {
      // the variable added in the SQL request will be found in the $_values of the doctrine record
      if (isset($this->_values['nb_tags']))
      {
        $this->nb_tags = $this->_values['nb_tags'];
      }
      else
      {
        /**
         * The number of tags has not been set in the SQL request
         * Doctrine will lazy load every tag and count them all.
         * This could be optimized by overwriting the createQuery method,
         * adding a left join to the tag table automatically in BlogPostTable.class.php
         * (beware, it can lead to unwanted side effects)
         */
        $this->nb_tags = $this->getTags()->count();
      }
    }
 
    return $this->nb_tags;
  }

Conclusion

So what have we achieved? Simple: we reduced the number of SQL requests in our admin gen from 22 to 2! One to retrieve the blog posts with the number of related tags, and the other by the Doctrine pager. Obviously, this trick isn’t restricted to admin generators, so think of the many situations where you can use it!

Filed under : Tips, doctrine, symfony — Benjamin Grandfond @ 16 h 08 min

11 April 2011

DevOps conference online!

My conference about adopting DevOps philosophy on Symfony projects is now online! You can see (and listen to) me speaking here: http://symfony.com/video/Paris2011/573

In this presentation you will see what I think is the philosophy behind the DevOps movement and how to start with the 4 important aspects of adopting DevOps:

  • Configuration Management with Puppet
  • Development on the production environment with Vagrant
  • Deployment automation with Fabric
  • Continuous deployment with Jenkins

If you are interested by the DevOps movement and you happen to be in Paris, come to our Paris-DevOps meetups. After the two first meetups hosted by Theodo, the meetup is now traveling to other locations. The next one will be held on the 4th of May at Xebia’s office. More info here: http://parisdevops.fr/

Filed under : Agile and DevOps, Theodo, symfony — Fabrice Bernhard @ 13 h 21 min

11 December 2010

Symfttpd: symfony’s long-awaited lightweight server

A project is never finished, and at Theodo we often have to quickly alter a project, in a small way. The issue is that the time required to having the project ready on a developer’s machine can be greater that the time required to do the the rest! But worse, setting up a project is boring.

This is why we decided to automate as much as possible the low added-value aspects of setting up a project.

There are usually four aspects to setting up a project:

  • Dependencies
  • Configuration files
  • A database
  • A web server, configured for the website, including the URL rewriting

Dependencies

In our case, it usually means only “Symfony, version 1.x”. For Python projects, a pip dependencies file is provided, and running a script will create a virtualenv and install the dependencies: no root access or hunting on the web is needed.

For Symfony projects, it’s easy to have all major versions at hand. However, these projects usually require multiple symbolic links to be created, in non-standardized places. Thankfully, we chose to adopt convention over configuration and only one line of configuration is actually needed for mksymlinks to do its magic.

If you chose to install Symfony other than in lib/vendor (likely with versions under 1.2), you can of course configure mksymlinks to handle it.

Configuration

The configuration of a project on the production servers and on the developer’s machines usually differ; database passwords, mail and database servers location, etc. The configuration files are therefore not versioned, only sample versions of them are, designed for a standard development environment.

A simple script can find the “sample” configuration files and create symbolic links (ideal for developers) or copies of these sample files. The developer shouldn’t need to alter these files to setup the project.

Running the script will for instance create a symlink from config/databases.yml to config/databases.sample.yml.

Databases

While recent versions of Symfony and Doctrine can create a database, older versions and Propel can’t. Also, database users and their credentials still have to be created, and it is really annoying to set them up. We use a simple script which will write a SQL script, and will try to run it using the credentials of ~/.my.cnf.

An alternative is to use SQLite, but it can quickly become a limitation for bigger projects.

The database is then filled with meaningful and useful fixtures.

The web server

Last but not least, the web server. Frameworks like Ruby on Rails or Django come with their own servers. These are not for production use, but are perfect for developer’s needs: no root access required, instant startup, no configuration… In the PHP world, nothing similar to be found. However, it is very well possible to configure and start a small server automatically.

That’s the job of symfttpd and its command spawn: run it and it will configure and start a lighttpd-based server, then tell you where you can view the project, and what applications are present.

The scripts handling the sample configuration and database are rather crude and around 50 lines of code; you should be able to write your own in no time if you are too impatient for a proper release of our own.

Let’s not forget documentation

While all of this works great, nothing replaces an INSTALL file with all the necessary steps, especially the unusual ones. It’s also nice to provide scripts to run all the required commands in one go; run it and your project is ready! (There’s still time to get a nice cup of coffee especially with our new awesome coffee machine.)

There is more to come

On the same principles, we use some of the tools mentioned above in our continuous integration platform, to be able to add any project in no time.

Symfttpd with its tools spawn and mksymlinks has been publicly released: https://github.com/laurentb/symfttpd. You can use them to accelerate the deployment of your development environment. As for the small scripts which automate the rest of our needs, we will surely release them in an upcoming post about continuous integration. More about it will follow, so be sure to subcribe!

Filed under : Theodo, symfony — Laurent Bachelier @ 0 h 19 min

2 December 2009

More with symfony: a whole chapter devoted to the sfFacebookConnect Plugin

December just started, and with it its usual christmas spirit, Santa Claus, happy children and…. the symfony advent calendar!

This year the symfony advent calendar is a collection of articles written by different symfony experts:

http://www.symfony-project.org/blog/2009/12/01/one-more-thing

and is already available as a book on Amazon!

http://www.amazon.com/exec/obidos/ASIN/2918390178

I had the chance to contribute and write an article on developing for Facebook with symfony. This was the perfect occasion to finally sit down and write 15 pages on the experience I gathered on this specific subject. I had already collected it in the sfFacebookConnect plugin but it was lacking documentation. Well here it is finally! At least on Amazon and in a few days as part of the new symfony advent calendar.

Enjoy and do not hesitate to make a critical feedback, the article will be included with the plugin and can still evolve a lot!

Filed under : sfFacebookConnectPlugin, symfony — Fabrice Bernhard @ 10 h 42 min

24 August 2009

A Facebook Connect plugin for symfony

As promised during the last symfony live conference, I finally release my current work on a Facebook Connect Plugin for symfony. It is inspired by the good sfFacebookPlugin by Jonathan Todd, which has however been unmaintained for quite some time. Since Facebook’s platform is evolving every week and my focus was not on the Facebook platform but on the Facebook Connect functionality, I decided to create this new plugin.

It is for the moment VERY beta. It is used in two projects, http://www.allomatch.com which is a symfony 1.0/propel project and another project on symfony 1.2/doctrine. It is therefore compatible with both Doctrine and Propel. However some issues remain concerning 1.0 and 1.2 versions regarding some options, the tasks for example.

For the installation, the README is a good start but FAR from complete. I invite you to browse through the code to understand the logic and comment on this post if you have any question regarding installation. This will force me to improve the README.

I intend to improve the documentation in the very near future, so if you are not in a hurry, please wait. However I have already received dozens of mails concerning the current status, so I release it for those who need to start a project using Facebook Connect right now.

Here is the link to the plugin:

http://www.symfony-project.org/plugins/sfFacebookConnectPlugin

And here the presentation made at the sflive conference:

http://www.symfony-live.com/pdf/sflive09fr/theodo-symfony-facebook.pdf

Filed under : sfFacebookConnectPlugin, symfony — Fabrice Bernhard @ 12 h 02 min

3 June 2009

sfEasyGMapPlugin 1.0.4 out

sfEasyGMapPlugin 1.0.4 is out and the good news is : the plugin is the 24th most used symfony plugin among the 457 available on http://www.symfony-project.org/plugins/ ! We are now 5 official developers, not counting all the developers I work with who contribute indirectly.

It all started because I was amazed by the success of the Phoogle library on the Internet despite its limited number of functionalities. And since almost all my projects involved a Google Map I wanted to create a plugin containing all the core functionalities I always reuse. Now I am happy to see the popularity of the plugin and am looking forward further possible developments that will continue in the spirit of including as many core functionalities of Google Maps-based application in the plugin.

New functionalities for the moment include :

  • More precise Mercator projections to convert GPS coordinates into Google Pixel coordinates and back GMapCoord::fromPixToLat, GMapCoord::fromLatToPix, etc.
  • Added the GMapBounds::getBoundsContainingMarkers(…) function
  • Added the GMap::centerAndZoomOnMarkers() function which enables to guess zoom and center of the map to fit the markers. Center is easy to guess. Zoom uses width and height of smallest bound, pixel width and height of the map and Mercator projection
  • Added tomr’s contribution: it is now possible to add multiple controls to the map
  • Added the GMapCoord::distance($coord1, $coord2) function which gives an estimation of the distance between two coordinates
  • Added the very useful function $gMap-> getBoundsFromCenterAndZoom(…) which enables one to calculate server-side the bounds corresponding to specific center coordinates, zoom, and map size. This is the equivalent of the client-side map.setCenter(…,…);map.setZoom(…);map.getBounds(); It uses the Mercator projection formulas as used by the Google Maps
  • A new function $gMapMarker->isInsideBounds($bounds)
  • A lot of unit tests
  • And two new samples

Please, feel free to suggest what you consider typical core functionalities of your Google Maps-based applications.

Filed under : sfEasyGMapPlugin, symfony — Fabrice Bernhard @ 0 h 49 min

13 March 2009

Symfony Google Maps API plugin : sfEasyGMapPlugin v1.0 is out!

sfEasyGmapPlugin is a very easy to use Google Maps API plugin for symfony, inspired by the Phoogle class… but better :-)

A very simple version has been available for a few months but I have now finally released the 1.0 version, with the following new features :

- it is now sf1.2 compatible straight out of the box
- it has some unit tests
- the GMap constructor now takes an array of parameters, which is much more flexible and also more in the symfony coding spirit (Warning : the modification of the GMap constructor should break your application if you used the prior version of sfEasyGMapPlugin)
- there are interesting functions concerning Bounds :
- smallest enclosing bound
- propel criteria “in bounds”
- homothety transformation
- zoomOut transformation
- there are interesting functions concerning conversion from/to lat/lng to/from Google’s pixel coordinates system. These can be very useful if you want to guess the bounds knowing only the center lat/lng, the zoom level and the map’s width/height in pixels. They involve a few mathematical formulas that were not so straightforward, (since you need to understand how Google’s projection works) so trust me, these functions are valuable, even if they only concern power users.

I have also developed a few doctrine-specific functions which are unfortunately not available yet because not generic enough. I will try to release them in the next version.

The official symfony page is here : http://www.symfony-project.org/plugins/sfEasyGMapPlugin

Please feel free to comment on this work in progress !

Filed under : sfEasyGMapPlugin, symfony — Fabrice Bernhard @ 16 h 16 min

6 March 2009

Integrate Wordpress into symfony

What better topic to start this technical blog about symfony than to talk about my experience of integrating Wordpress into symfony !

I was looking for a nice blogging solution for symfony, and all I found was a very simple plugin and a lot of people encouraging me to build my own blog. Even though it is a nice exercise, my philosophy is to not reinvent the wheel. Wordpress is surely the best free blogging tool available, so I preferred to spend time integrating it into my symfony application than to create yet another sfVeryEasyBlogPlugin.

Integrating Wordpress into symfony can be done in three steps : (more…)

Filed under : sfWordpressPlugin, symfony — Fabrice Bernhard @ 16 h 19 min