All articles, tagged with “django”

Short urls — rev=canonical

2009, April 12

Tags: ,

Just yesterday my feed reader fetched Simon Willison’s article about relatively new method of url shortening.

The idea is that url shorteners in fact are not very good thing, especially from URL term of life point of view. What if tinyurl.com is going to die? A lot of URLs will appear unusable, even if their destination is still alive. And discussion gave birth to thought that it could be nice to allow sites to specify their own short urls, even using their own url shorteners.

There is site with specification (if it can be named so), in short — you should specify it in your <head>...</head> section with such link:

<link rev="canonical" href="short-url-here">

In his article Simon tells about how he has implemented rev="canonical" in it’s own blog — I like his solution for not saving anything in database. ;) But hardcoding actual types of content is not that cool, so I’ve done small Django application — revcanonical, whose task is to generate and redirect such short links.

Every url is a two strings joined by a dot — it’s a base62-coded1 identification numbers of content type and object in database. So you can use that application for any object without any configuration. README has instructions on how to install and use application.

Of course, functionality is already incorporated into Byteflow. ;-)

  1. Digits, big and small latin letters ↩

0 comments »

FirePython — no prints?

2008, November 24

Tags: , ,

While I’m developing some web application I almost never use any debuggers or supporting tools: in 90% of cases usual print variable is enough for understanding a trouble. Of course, there are some very complicated cases, when I do import pdb; pdb.set_trace().

But I was pointed at very cool thing yesterday — FirePython. This thing consists of two parts — small library on python and plugin for Firebug. This tandem is engaged with very useful business — it displays all python logged activity1 in Firebug tab.

continue reading (35 comments) »

Django performance: Apache vs Nginx

2007, December 01

Tags: , , , , ,

Just a week ago I have compared performance and stability of nginx working with Django through mod_wsgi and fastcgi. Comparison revealed that performance with fast local DB (i.e. time of DB work is negligibly small) is almost identical and all the difference is in used resources.

Today at last I have time to look, what will be, when database is located at other computer and WiFi connection between application and DB worsening situation. :-) I.e. sense of my today’s actions is looking at situation, when DB is slow (this is interesting for me after Manlio’s commentary in previous post). Additionally I wanted to look at Apache’s behavior.

Before doing that I runned lightweight test (ab -n 100 -c 20) on Apache (with local database), which showed up that Apache don’t want to use my two cores. :-( Nor prefork, neither worker don’t used second core and time between requests was around 28 ms, which is two times to nginx’ 14 ms. Logic thoughts said that heavy-weight Apache in any case is slower than nginx — second core will not improve performance in two times (so says nginx’ 24 ms when working with one core :-).

Further PostgreSQL was launched at another laptop. Apache was tested in both versions, prefork and worker and displayed no difference, so I’ve stopped on worker.

First — ab -n 1000 -c 20:

  • Apache — ~37 ms. Very stable, difference in time between 5 runs was lesser than 1 ms.
  • nginx + mod_wsgi — ~50 ms. Expected (as Manlio said, nginx is blocked while working with application, which is blocked by request to DB).
  • nginx + fastcgi prefork — ~28 ms. I can’t believe it is so faster than Apache! :-)

Second test — ab -n 3000 -c 500 — is not very distinguished from first. Apache and fastcgi — same result, mod_wsgi raised to 55 ms.

At that moment I thought that mod_wsgi is only suited for application without database (or when delays for its work is negligibly small). But, thought over, I made knight’s move — raised number of workers in nginx. :-) With 16 workers (tested after — 8 is enough) and 500 concurrent connections delay between answers is 28 ms. Now I can believe in fastcgi result and Apache is heavy and fat, as usual. ;-) Thought, every nginx’ process, which works with Django, eats around 15 mb of RAM. However apache and fastcgi (threaded, forked) want bigger amount of memory.

Everyone can do summary for himself, the only one thing I can say unambiguously — nginx + mod_wsgi is very interesting combination.

0 comments »

Nginx + Django: mod_wsgi vs FastCGI

2007, November 24

Tags: , , , ,

(Translation of my previous post).

Yesterday I have finally sorted out my trouble with compilation of nginx’ mod_wsgi and got it working (how much means slash in our life ;-).

Configuration to get Django application running is quite simple — django.wsgi, which is used for Apache’s mod_wsgi, fits perfectly. I’ll not describe all configuration, because code is unstable and needs testing and is’s contra-indicated to run it on production now.

Lets do the fun — testing. :-) Simple page, 15 queries (PostgreSQL on same pc), 2 workers for nginx (I have tried 3 and 4 — they both are slighty slower, for 2-4 ms). Two variants of testing queries (ab -n 1000 -c 20 and ab -n 10000 -c 500) and three variants of servers (mod_wsgi, prefork fastcgi, threaded fastcgi). Hardware (this is not so interesting, because performance is interesting only in comparison, but why not to show it?) — Core 2 Duo T7300 and 2 Gb of RAM.

First — ab -n 1000 -c 20 (around of 10-12 runs for every variant):

  • mod_wsgi takes 14.2-14.3 ms per query, quite stable performance
  • prefork fastcgi takes 12.5-16.5 ms (mostly near of 12 ms, but raises from time to time), eats bigger amount of RAM — I have an xmobar1 showing usage of RAM and mod_wsgi has a two-three percents (percent is 20 megs) lesser usage
  • threaded fastcgi takes 24-25 ms per query — it uses only one core of CPU. I have tried to get upstream working in nginx — it works, but uses only 1 process for some reason :-(

I.e. in most cases for such load FastCGI is slighty faster (and uses somewhat bigger amount of resources). But… lets go further? ;)

Second — ab -n 10000 -c 500 (here I got 3-4 runs for every variant):

  • mod_wsgi takes 13-14 ms. Pretty stable result. Each worker consumes 21/15 megs of RAM (VSZ/RSS)
  • prefork fastcgi takes 15.5-17 ms per request, but it runs from 40 to 50 serving instances, each consuming 16-20 VSZ (10-13 RSS) megs of RAM! xmobar says that usage raises up to 50% (by leaps, usually around 45-48%) — compare to 33% for mod_wsgi! Additional 300 mbytes of RAM.
  • threaded fastcgi — most “interesting” variant. With this level of concurrency it just dies. With -c 100 — dies. It lives with -c 50 with slighty lower speed than with -c 20, but process eats near of 300 mb of VSZ.

What I can say here? Lets wait for stable mod_wsgi release! :-) Thanks, Manlio, for this piece of code. :-)

1 —

xmobar — small statusbar, written on Haskell. Displays system state and other various information

1 comment »