BlenderPeople Development

Tracking the development of the BlenderPeople script suite.

Thursday, December 22, 2005

SQLite/Python Version Down The Toilet

I've finished the prototype version of BlenderPeople that uses SQLite in combination with local Python lists and dictionaries to mimic the functionality formerly achieved with MySQL. The advantage is that it is almost zero configuration. Download and install Python 2.4, if you haven't already. DLaI SQLite for Python. Run BlenderPeople. It works.

The problem is that it's really slow. SQLite lacks "advanced" math functions like square root, but more importantly, it does not have trig functions. Trig is an absolute necessity for BlenderPeople, because you often want to select, for example, "the nearest opponent to my current Actor, within his field of view." That requires both square root and trig (BTW, I know you can distance search on just the square, and I do, but there are other times when you don't want to pre- or post-process your query results). One nice thing about the SQLite Python implementation is that it's a piece of cake to add those functions. So I did. You simply send a query to SQLite defining the function name and it's reference in your Python code. Then make a Python function that does what you want.

I did all of that for the necessary math and ran it. It was slow like a two-legged dog. Rounds of calculation that were taking around one second in the MySQL implementation were taking as long as fifteen seconds with SQLite. Actually, it was taking almost ninety seconds per round when I first implemented, but that was due to the way SQLite writes every transaction to disk immediately, and you have to manage your connection commits more carefully than with MySQL.

So, I moved on to step 2 of the project, which was using straight Python to do a lot of the searching and sorting work. If going from Blender->Python->SQLite->PythonMath was too slow, I figured I'd just do all the math right in Python and skip the superfluous connections. Where before, I would access the database several times for each Actor in a turn, writing changes back immediately, I would now access the DB only once per turn, do all searching/sorting/filtering on lists directly in Python, then write the result back at the end of the turn. That should remove almost all bottlenecks thrown up by SQLite.

It turns out, though, that SQLite wasn't the problem. It was Python. Round times using the pure Python method were comparable to the previous SQLite times. Using lists in Python only shaved about 10% off the slow SQLite implementation.

Unfortunately, I'm going to have to abandon this project. For a 1,000 Actor simulation, I can get times of around thirty seconds per round on my system (Athlon XP 1800 w/512MB RAM), which is pretty outdated by today's standards. That means the Python/SQLite combo version would give me times of around five minutes per turn. That's way to slow to be useful. One of the cool things about BlenderPeople is that it's fast enough that you can sit there, watch it develop, adjust some settings, etc., and continue the simulation. With five minutes rounds, that becomes impractical and significantly less useful.

My next step is in trying to streamline the MySQL installation and configuration process, which has gone from fairly simple when I started BlenderPeople, to a laborious hulking nightmare these days.

posted by Roland  # 4:07 AM
Comments: Post a Comment


02/01/2004 - 02/29/2004   04/01/2004 - 04/30/2004   05/01/2004 - 05/31/2004   06/01/2004 - 06/30/2004   07/01/2004 - 07/31/2004   08/01/2004 - 08/31/2004   09/01/2004 - 09/30/2004   11/01/2004 - 11/30/2004   12/01/2004 - 12/31/2004   01/01/2005 - 01/31/2005   02/01/2005 - 02/28/2005   06/01/2005 - 06/30/2005   09/01/2005 - 09/30/2005   10/01/2005 - 10/31/2005   11/01/2005 - 11/30/2005   12/01/2005 - 12/31/2005   01/01/2006 - 01/31/2006   03/01/2006 - 03/31/2006   04/01/2006 - 04/30/2006   05/01/2006 - 05/31/2006   06/01/2006 - 06/30/2006   07/01/2006 - 07/31/2006   08/01/2006 - 08/31/2006   09/01/2006 - 09/30/2006   10/01/2006 - 10/31/2006   11/01/2006 - 11/30/2006  

This page is powered by Blogger. Isn't yours?