Using INNER JOIN/SELF JOIN to allow for smaller indexes.

by Allan Svelmøe Hansen May 15, 2009 16:26

In databases, I often need foreign keys in my tables, because I’ll want to use them to select content out from my tables. However this can often result in either bad index utilization in the selection or making additional indexes based on the foreign key(s) and the content I need to select out.
This in turn can result in ending up with “many” indexes and sometimes many big indexes.

However a method can be to make smaller indexes and use an INNER JOIN to join into the table an extra time.

I’m going to show the pattern with a relative simple example to illustrate, because it is about the pattern more so then the specifics layout, content and size of the table.
It is just a pattern/technique to keep in mind and have in the SQL toolbox.
Suppose you have two tables of a similar pattern to this:

Tables for self join example
(click for larger size)

I’ve let the SQL Server create my clustered indexes based on the primary key, which means I have a clustered index over my primary key(s).
When having to look up rows in JoinOne based on the foreign key, it’ll often look like this:

SELECT 
FROM 
JoinOne T1
WHERE FKOne <VALUE>
  

Because I’ve added no other indexes to the tables, I’ll get an Index Scan or Table Scan when running the above query, which - as we - know is not a good thing most of the time.
This usually leads to the creating of a second index which indexes over my foreign key.
However if the table is large, and if I’ll need to extract many/most of the columns (like in my example with SELECT *), it can mean I’ll have to make either a complete index or one with many included columns, just ordered by my foreign key first.

If I make a small index with my foreign key and my primary key second, I can use this to join into my table again, and then after that use my primary key.
This is an index to illustrate:

Index for self join example 
(click for larger size)

Then I can make the following query:

SELECT T2.
FROM 
JoinOne T1
INNER JOIN JoinOne T2 
ON
 
T1.PKOne 
T2.PKOne
 
AND T1.PKTwo 
T2.PKTwo
 
AND T1.PKThree 
T2.PKThree
WHERE T1.FKOne <VALUE>
  

When looking at its execution plan, it’ll show two index seeks instead of my previous index scan. This way I have a small index as possible, but maintain seeks in my execution.
Of course there are some considerations one need to take with this pattern. Firstly – if the tables are “small”, then the scan in itself might be alright, or the overhead of keeping a complete index for the foreign key is neglible.

However the advantage of the pattern is that I can have smaller indexes on (very) large tables, which means less overhead when inserting/updating – but still have only index seeks in my selections execution plans.
It is a useful technique in my opinion – when used at the right times, which of course is on a case by case evaluation as always.  

Bookmark and Share DotnetKicks dotnetshoutout

IN versus INNER JOIN

by Allan Svelmøe Hansen May 11, 2008 19:22

I decided to do some testing on inner joins versus the in syntax, meaning what to use if I need to select the content from one table out, based on content in another table.
So what I did was to create a couple of tables:
CREATE TABLE TestOne
(
 
ID1 INT NOT NULL,
 
Value1 NVARCHAR(255) NOT NULL
)
GO

CREATE TABLE TestTwo
(
 
ID2 INT NOT NULL,
 
Value2 NVARCHAR(255) NOT NULL,
 
ID1 INT NOT NULL
)
GO 
 

I then populated them with some test data (1000 rows for the first and 10000 rows for the second of pointless data). Note that I have added no indexes or restraints of any kind as I wanted the testing to be as base as possible. So then I basically pulled data out from TestOne filtered by ID1 having to be in TestTwo.
Like this:

SELECT *
FROM TestOne T1
WHERE
T1.ID1 IN (SELECT ID1 FROM TestTwo)
GO

SELECT T1.*
FROM TestOne T1
INNER JOIN TestTwo T2 ON T1.ID1 T2.ID1
GO
 

Then I looked at the estimated query plans and was a bit stupefied by the result.

(click to enlarge)
Estimated Execution Plan For IN versus INNER JOIN for first and second query
There was a massive difference between the two, and this shows that it is actually better to just use the IN operator instead of INNER JOIN when filtering. Of course if you actually need the content of the filtering table then you need to join, but just filtering – then IN looks clearly superior.
This result was made even more visible when I added a third table (similar to the second one) and joined that into the equation.

Then I wanted to make some more complex queries, because it is rare that I write such simple queries at work. But of course now it gets more “thought out” and fictive now, and I should have done this with real life work data to get a better indication. But this’ll have to do for now :)
Again note that no indexes, keys, restraints or anything are made.

So I populated the database two more tables (Four and Five) and these additional tables and data very similar to how I populated the first three.

Then I made the following queries

SELECT T1.*
FROM TestOne T1
WHERE T1.ID1 IN (
  
SELECT ID1 FROM TestTwo T2
  
WHERE ID2 IN (
   
SELECT ID2 FROM TestFour T4
   
WHERE ID4 IN (
    
SELECT ID4 FROM TestFive T5
   
)
  )
 )
GO

SELECT T1.*
FROM TestOne T1
INNER JOIN TestTwo T2 ON T1.ID1 T2.ID1
INNER JOIN TestFour T4 ON T2.ID2 T4.ID2
INNER JOIN TestFive T5 ON T4.ID4 T5.ID4
GO   
 

What I select out is TestOne filtered by TestTwo which is filtered by TestFour which again is filtered by TestFive.
This gave a similar result as before, but with a smaller margin.
Now it was 41% versus 59% in favor of the IN syntax.
(click to enlarge)
Estimated Execution Plan For IN versus INNER JOIN for third query
(click to enlarge)
Estimated Execution Plan For IN versus INNER JOIN for fourth query
This means that we are now entering a more case-by-case area where one can’t say for sure which method is best. Also personally – I would say that the INNER JOIN syntax makes for a more readable query, but that of course is personal preference.
But also imagine that I suddenly had to get the information from TestFour as well, for example – that could make quite a nasty query if I tried to use the IN syntax, whereas I could simply change the SELECT in the latter query to SELECT T1.*, T4.* and then done.

Then I tried to add some keys and indexes, but that did not change the result.

So in conclusion, if making simple filtering, then the IN syntax looks vastly superior against the INNER JOIN syntax, but more complex queries and if you need the data from the filtering tables, makes the conclusion more gray and would need a case-by-case conclusion.

Another time, I’ll try to dabble in using LEFT and RIGHT JOINs as filtering against the NOT IN syntax.

 

Bookmark and Share DotnetKicks dotnetshoutout

Powered by BlogEngine.NET 1.6.1.0
Theme by Mads Kristensen | Modified by Mooglegiant

About:
Allan Svelmøe Hansen

My real name is Allan Svelmøe Hansen.
I live in Denmark, where I work as a developer for hedal:kruse:brohus using SQL Server and the .NET framework since 2004.
My primary fields of expertise is back-end data integration, database design and optimization.


       View Allan Svelmøe Hansen's profile on LinkedIn     

Disclaimer

The opinions expressed herein are my own personal opinions and thoughts and does not represent my employers view in any way, nor are my results guaranteed for all situations.
Content is presented "as is", with no warranty.