Relevance of the Ebay-Paypal investment


Relevance of the Ebay-Paypal investment

This content was originally published in February 2011.

Simply, Magento has gone from strength to strength in the past 12 months in directions that the developers and the community has been largely happy with. If this sort of announcement had taken place on the date of investment, Ebay/Paypal would have a terribly steep uphill battle convincing developers that they had the community’s best wishes in mind, but now had an opportunity to do so by example.

I think we have definitely benefitted from the investment as the community and if we extrapolate, we can assume minimal corporate interferences. Yet Extrapolation is a dangerous thing in the hands of the inexperienced and I am not calling myself a statistician. Yoav Kutner and Rhonda Rondeau had expressed a commitment to working closer with developers over the upcoming year. Their head of development even spent six months in support which I a step in the right direction. More internal processes with the management of the Magento platform in general will be opening up soon, so I see to need to Chicken Little this announcement.


Switching your Magento development environment from Apache to Nginx


Switching your Magento development environment from Apache to Nginx

NOTE: This was published originally in 2011 and the instructions may be outdated by now. A new post is coming soon. 

While at the Magento Imagine conference a couple weeks ago I learned that the entire Magento site and its demo sites all run on nginx instead of Apache. I was aware of that several alternatives to Apache existed and served various niches especially when it comes to delivering static HTML files at speed, however never really investigated them properly. Zeus and lighttpd were always just a blur on the horizon.

Seeing that I was consuming lots of information about node.js in the last couple weeks, I unavoidable came across infamous claim that node.js was faster than nginx by Ryan Dahl the author of node.js. Seeing that I was completely out of the loop as to how important this is, I investigated nginx a little more and realized that it was indeed a formidable server after PHP-FPM and APC were installed. I took a couple weeks, running tests off and on trying to see how much of an advantage there was to using nginx, PHP_FPM and APC. Given that now I have an awesome fast dev rig, the speed for the client showed little change in my tests even running something like Magento. But I found a difference when I started putting the servers on virtual machines via Virtual Box with limited specs (as close as possible to the t1.mirco AMI Amazon has so kindly offered me for one year) Wanting to play with new technologies, I stopped after three tests and decided that I was on board with nginx, especially since it was adopted by WordPress and Magento.

I intend to do some more rigorous testing and security analysis before launching a production site on this combo because making the move from Apache is a huge step. But first steps were getting my local versions of Magento Community Edition and running on nginx. Yes I know I mentioned earlier that nginx made little difference when running on a brand new dev rig so why am I doing this? I firmly believe in building your software against as close as possible technology stack as your final deployment. When I worked as the Solution Architect on the Paramount Pictures/Seagate online store, all the offshore developers were building Magento stores on Windows. We had nightmares when it came to weekly deploys yet I couldn’t change the policy of an organization half way around the world. So just to be safe, if I am going to deploy onto nginx, I am going to code on an nginx server.

I usually have multiple virtual hosts running on my dev rig to manage development of extensions against multiple versions of Magento. There is also a custom build and deployment system in the back there holding everything together with git, but that is another post. The instructions below are only really intended to move a Ubuntu 10.10 development machine to nginx away from Apache. I would advice against blindly copy-pasting what you see here into your production machine. It will probably work but I am not responsible if anything breaks.

First step is to shutdown Apache

sudo service apache2 stop

I usually try to remove all the unused packages from my system if I am not going to use them. That is why my build/deploy system is built on shell scripts as opposed to anything fancy like Capistrano, Maven etc. So the next step is removing Apache from your system. I resorted to Synaptic package manager for this because a simple sudo apt-get remove apache2  didn’t remove all the packages. Here is a list of all packages I ended up removing.

sudo aptitude remove apache2 apache2.2-common apache2.2-bin apache2-utils apache2-mpm-prefork libapache-mod-php5

At this point Synaptic package manager insisted that I install the php5-cgi package so I just went along with it assuming that I will be able to remove it later on, but also aware that PHP-FPM works with PHP in a CGI context so I left it alone.

That should take care of Apache.  The next step is to install nginx and start it up

sudo aptitude install nginx
sudo service nginx start

At this point nginx should have gone to your /var/www folder and carefully dropped in a folder callednginx-default in there to not disturb your existing default folder if there is one. Absentmindedly it then forgets where it put its own docroot and points is default docroot to /var/www so when you hit your localhost in a browser, you see the default nginx 404 error page. You can drop in an index.html into/var/www to see what nginx will do.

The next step is install PHP-FPM which is a bunch of binaries that attach to PHP and run a service on your localhost on port 9000. Basically nginx queries this service for the PHP compiled data to send to the browser.

sudo aptitude install php5-fpm
sudo service php5-fpm start

If you are getting “ [WARNING] [pool www] pm.start_servers is not set” and you are OCD you can set the value in /etc/php/fpm/pool.d/www.conf on line 73.

APC is a PHP opcode cache that can cache the interpreted PHP script and even variables. From my testing it provided almost negligible change on a dev machine but it may run on a public server so again I installed it here. Should there be cache refresh issues we will catch it in development as opposed to launch. Zend has a good starter article on APC if you want to learn more.

sudo aptitude install php-apc

Php should be up and running but we haven’t told nginx about it yet. There are configurations for doing so which you can get from this How To Forge tutorial, but we are not going down that path today. We are going to set up two virtual hosts which point to two installs of Magento CE and So I am going to make two new files in /etc/nginx/sites-available/ , one called local1420 and the other called local1501 . Here I switch over to the wisdom of the Magento Wiki page on nginx with a configuration for the nginx.conf and the individual site configurations.

I have found that the nginx.conf setting works fine, but I had to tweak the individual site configs to work properly. When I have confirmed with someone smarter than me that I am doing it as right as possible I will probably modify the Magento Wiki to reflect the changes I made. For now I will post my file here so you can compare.

The major changes I made was a result of wanting to avoid and just sticking with so I removed lines 4-10 of the Magento suggested configurations. Beyond that I was not using HTTPS on my local dev machine, so I commented out fastcgi_param  HTTPS$fastcgi_https;. Also seeing that I was not developing against a multistore environment, I commented out fastcgi_param  MAGE_RUN_CODE default; and fastcgi_param  MAGE_RUN_TYPE store;. The last change I made was to include /etc/nginx/drop.conf which I will describe at the end of the article.

listen 80;
root /var/www/;
location / {
index index.html index.php; ## Allow a static html file to be shown first
try_files $uri $uri/ @handler; ## If missing pass the URI to Magento's front handler
expires 30d; ## Assume all files are cachable
## These locations would be hidden by .htaccess normally
location /app/ { deny all; }
location /includes/ { deny all; }
location /lib/ { deny all; }
location /media/downloadable/ { deny all; }
location /pkginfo/ { deny all; }
location /report/config.xml { deny all; }
location /var/ { deny all; }
location /var/export/ { ## Allow admins only to view export folder
auth_basic "Restricted"; ## Message shown in login window
auth_basic_user_file htpasswd; ## See /etc/nginx/htpassword
autoindex on;
location /. { ## Disable .htaccess and other hidden files
return 404;
location @handler { ## Magento uses a common front handler
rewrite / /index.php;
location ~ .php/ { ## Forward paths like /js/index.php/x.js to relevant handler
rewrite ^(.*.php)/ $1 last;
location ~ .php$ { ## Execute PHP scripts
if (!-e $request_filename) { rewrite / /index.php last; } ## Catch 404s that try_files miss
expires off; ## Do not cache dynamic content
# fastcgi_param HTTPS $fastcgi_https;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
# fastcgi_param MAGE_RUN_CODE default; ## Store code is defined in administration > Configuration > Manage  Stores
# fastcgi_param MAGE_RUN_TYPE store;
include fastcgi_params; ## See /etc/nginx/fastcgi_params
include drop.conf;

The last bit of configuration I added was advised by Karl Bessing  Basically it involves creating a short file in /etc/nginx/ called drop.conf with the following code and including it in the server section of your virtual hosts. You can read more about what it does on his site.

location = /favicon.ico { access_log off; log_not_found off; }
location = /robots.txt { access_log off; log_not_found off; }
location ~ /. { deny all; access_log off; log_not_found off; }

And with that you should have all your configuration files all ready. Next step is to link in your available sites to your enabled sites. Usually you would use a2ensite for something like this in Apache, but I was unaware if anything similar existed for nginx, so I just linked them in manually.

sudo ln -s /etc/nginx/sites-available/local1420 /etc/nginx/sites-enabled/local1420
sudo ln -s /etc/nginx/sites-available/local1501 /etc/nginx/sites-enabled/local1501

Restart nginx and php5-fpm and you are done.

sudo service nginx restart
sudo service php5-fpm restart

If you are installing Magento from scratch you can check the box displayed during setup labelled “Web server Apache rewrites”, because it works just fine. If you have any questions or improvements, feel free to leave a comment.


Javascript Master Class


Javascript Master Class

I have read Javascript: The Good Parts* cover to cover multiple times. ( Well OK lately, its been BOF to EOF on my iPad ) When someone is as resolute and pragmatic as Douglas Crockford is to a technology as widely misunderstood and reviled as Javascript, human beings take notice. I believe it is inherent in our evolution to take notice at those who are that tenacious without being bullheaded.

Since I was learning node.js during my two month work hiatus, I thought it would be a good time to brush up on good Javascript style and eventually I started poking around his book once more, and lo, I found out that Safari Books Online** had the entirety of his Javascript Master Class available for streaming. Starting out a 5+ hour tutorial at almost midnight would not count as one of the smartest things I have done in my life, but it was worth it. Unfortunately I had just gotten past my second wind when he hit Pseudoclassical Inheritance in Javascript. By far that is one of the most convoluted processes I have ever had the pleasure to try grokking at 2am. I have had nightmares for weeks because I called it quits for the night at that point.

In all honesty, the videos are phenomenal. If you are even considering being part of the future of web application development I suggest you look at the videos. Think of it like this, if someone can preface their highly detailed and technical talk about Javascript by starting with the technology of puchcards, then that person is an Alpha in his field and you should listen to him when he speaks. I was completely taken in from the first video. Reading his book on the subject matter and listening to him speak about it are two completely different things. Also I believe he goes into a little more detail in the videos.

Another Douglas, Douglas Engelbart

Another Douglas, Douglas Engelbart

Many parts of the videos I have returned to, and reviewed to make sure I absorbed all the details, but that one section about Pseudoclassical Inheritance I recoil in horror every time I think about trying to understand it. I still have a bunch of notes of things to look into, they are mostly little sparks that lit up in my head while I was learning things about Javascript I still need to try. (One of which, was “The mother of all demos” by Douglas Engelbart which I am embarrassed to admit I have not looked at yet)

Having knowledge of the history of technology I believe is a very healthy practice. Especially with the headlong rush we technologists have found ourselves in, there is always something stirring in me to step back and look at the big picture. I believe that by studying where we have tread to get where we are technologically gives you a better understanding and appreciation for the shortcoming and awesomeness of technology today. But I think critically it gives you an opportunity to cut through the stampede of developers and find new ways to direct technology in the future. That I believe is one of Mr. Crockford’s strengths.

*Disclaimer: The Amazon referral code (if there is one here) is Mr. Crockfords. I make nothing if you click on that link.
** I am still angry at Safari Books Online after their stupid iPad app fiasco, so no link for them. Half-assed software development is a cardinal sin in my eyes. I will rant about this later.