NHibernate Forge
The official new home for the NHibernate for .NET community

Bulk Data Operations With NHibernate's Stateless Sessions

Note: this was originally posted on my own blog

In my previous post, i showed how you can configure NHibernate to batch create/update/delete statements and what kind of performance benefits you can get from it. In this post, we're going to take this a bit further so we can actually use NHibernate in bulk data operations, an area where ORM's traditionally perform pretty badly.

First of all, let's get back to our test code from the last post:

            var testObjects = CreateTestObjects(500000);

 

            var stopwatch = new Stopwatch();

            stopwatch.Start();

 

            using (ITransaction transaction = Session.BeginTransaction())

            {

                foreach (var testObject in testObjects)

                {

                    Session.Save(testObject);

                }

 

                transaction.Commit();

            }

 

            stopwatch.Stop();

            var time = stopwatch.Elapsed;

The only thing that changed since the previous post is the amount of objects that are created. In the previous post we only created 10000 objects, whereas now we'll be creating 500000 objects. The batch size is configured like this:

    <property name="adonet.batch_size">100</property>

This means that NHibernate will send its DML statements in batches of 100 statements instead of sending all of them one by one. The above code runs in 2 minutes and 24 seconds with a batch size of 100. However, if we use NHibernate's IStatelessionSession instead of a regular ISession, we can get some nice improvements.

First of all, here's the code to use the IStatelessSession:

            var testObjects = CreateTestObjects(500000);

 

            var stopwatch = new Stopwatch();

            stopwatch.Start();

 

            using (IStatelessSession statelessSession = sessionFactory.OpenStatelessSession())

            using (ITransaction transaction = statelessSession.BeginTransaction())

            {

                foreach (var testObject in testObjects)

                {

                    statelessSession.Insert(testObject);

                }

 

                transaction.Commit();

            }

 

            stopwatch.Stop();

            var time = stopwatch.Elapsed;

As you can see, apart from the usage of the IStatelessSession instead of the regular ISession, this is pretty much the same code. With a batch-size of 100, this code creates and inserts the 500000 records in 1 minute and 26 seconds. While not a spectacular improvement, it's definitely a nice improvement in duration. The biggest difference however is in memory usage while the code is running. A regular NHibernate ISession keeps a lot of data in its first-level cache (this enables a lot of the NHibernate magical goodies). The IStatelessSession however, does no such thing. It does no caching whatsoever and it also doesn't fire all of the events that you could usually plug into. This is strictly meant to be used for bulk data operations.

To give you an idea on the difference in memory usage, here are the memory statistics (captured by Process Explorer) after running the original code (with the ISession instance):

And here are the memory statistics after running the modified code (with the IStatelessSession instance):

Quite a difference for what is essentially the same operation. We could even improve on this because the code in its current form keeps all of the object instances in its own collection, preventing them from being garbage collected after they have been inserted in the database. But i think this already demonstrates the value in using the IStatelessSession if you need to perform bulk operations. Obviously, this will never perform as well as a bulk data operation that directly uses low-level ADO.NET code. But if you already have the NHibernate mappings and infrastructure set up, implementing those bulk operations could be cheaper while still being 'fast enough' for most situations.


Posted oct 30 2008, 09:28 p.m. by Davy Brion
Filed under: , ,

Comments

Fabio Maulo wrote re: Bulk Data Operations With NHibernate's Stateless Sessions
on 10-31-2008 14:54

Another way for BulkUpdate without use ADO.NET directly is IQuery.ExecuteUpdate() (so far available only for native SQL).

sqhua wrote re: Bulk Data Operations With NHibernate's Stateless Sessions
on 02-01-2009 1:36

我發現測試數據要超過5W才能體現無狀態Session的優勢:

NHibernate.IStatelessSession 無狀態 = Sessions.GetStatelessSession();

           NHibernate.ISession 有狀態 = Sessions.GetSession();

           int testNumber = 100000;

           Stopwatch stopwatch = new Stopwatch();

           IList<Test> list = new List<Test>();

           for (int i = 0; i < testNumber; i++)

           {

               Test t = new Test();

               t.ID = i;

               t.Name = "批量加载" + i.ToString();

               list.Add(t);

           }

           stopwatch.Start();

           NHibernate.ITransaction tran = 無狀態.BeginTransaction();

           foreach (Test t in list)

               無狀態.Insert(t);

           tran.Commit();

           stopwatch.Stop();

           lb_insert.Text = "无状态插入:" + stopwatch.ElapsedMilliseconds.ToString();

           無狀態.Close();

           //list = NHHelper<Test>.GetIList("from Test");

           //無狀態 = Sessions.GetStatelessSession();

           //stopwatch.Reset();

           //stopwatch.Start();

           //foreach (Test tmp in list)

           //    無狀態.Delete(tmp);

           //無狀態.Close();

           //stopwatch.Stop();

           //lb_delete.Text = "无状态刪除:" + stopwatch.ElapsedMilliseconds.ToString();

           list.Clear();

           for (int i = 0; i < testNumber; i++)

           {

               Test t = new Test();

               t.ID = i;

               t.Name = "批量加载" + i.ToString();

               list.Add(t);

           }

           stopwatch = new Stopwatch();

           stopwatch.Start();

           tran = 有狀態.BeginTransaction();

           foreach (Test t in list)

               有狀態.Save(t);

           tran.Commit();

           stopwatch.Stop();

           有狀態.Close();

           lb_insert_old.Text = "有状态插入:" + stopwatch.ElapsedMilliseconds.ToString();

           //NHibernate.ISession s2 = Sessions.GetSession();

           //stopwatch.Reset();

           //stopwatch.Start();

           //有狀態 = Sessions.GetSession();

           //tran = 有狀態.BeginTransaction();

           //int r = 有狀態.Delete("from Test");

           //tran.Commit();

           //有狀態.Close();

           //stopwatch.Stop();

           //lb_delete_old.Text = "有状态刪除:" + stopwatch.ElapsedMilliseconds.ToString();

sqhua wrote re: Bulk Data Operations With NHibernate's Stateless Sessions
on 02-01-2009 1:39

如果只有1K或者更少的對象要處理,無狀態的Session比有狀態的還有慢大概2倍的時間

Tuna Toksoz's weblog wrote My Thoughts on ORM comparisons
on 08-21-2009 13:27

My Thoughts on ORM comparisons

Tuna Toksoz wrote My Thoughts on ORM comparisons
on 08-21-2009 13:27

I am a bit late to the party but there we go. Last several days, there have been a hot debate on the

Tuna Toksoz's weblog wrote My Thoughts on ORM comparisons
on 02-22-2011 20:24

My Thoughts on ORM comparisons

Powered by Community Server (Commercial Edition), by Telligent Systems