Issues when migrating stored procedures to native compiled

Recently, I started working on a project to migrate all disk based tables to InMemory and regular stored procedures to native compiled stored procedures.

While migrating stored procedures, I encountered several issues, that needed code changes. The native compilation advisor will alert on the issues that needs to be addressed before making the stored procedure native compiled.

This is not complete list but some of the issues I came across in my project

1. Cannot use Select * .
2. Cannot refer user defined functions.
3. Cannot refer tables without schema name.
4. Cannot use CASE expression, Choose, IIF.
5. Cannot use functions such ‘%’,Like.
6. Cannot have debug statements like Print.
7. Cannot have Create\Drop\Truncate Table.
8. Cannot use table variables.(use inmemory table variable)
9. Cannot refer objects in other databases.
10. Cannot use Linked Server Queries.
11. Cannot have explicit Begin Transaction\Commit\Rollback.
12. Lock Hints are not allowed.

I will add more exceptions here as I come across them.

Advertisement

InMemory Tables Pointers

Quick Notes on InMemory Tables:

1. Inmemory table cannot be truncated.
2. Identity value cannot be reseeded for InMemory table columns.
3. Indexed view cannot be created on Inmemory table (Views can be created on InMemory tables).
4. DBCC CheckTable does not work on InMemory Table. DBCC CheckDB skips inmemory tables.
5. There is no need to do index defragmentation for Inmemory table indexes.

(08/03/2016)Update: Few more pointers on InMemory Tables:

1. Identity value cannot be increment by more than 1 value.
2. InMemory Tables cannot be used in cross-database transactions. However, Inmemory table variables can be used for cross-database transactions.

Example:  The below query does not work for InMemory Table

Use TargetDB
Insert into MyInMemoryTable
Select * from SourceDB.dbo.MyDiskBasedTable

However, we can define the inmemory table variable(create a Inmemory Data Type and the table variable is created with this data type) and can use it.

Use TargetDB
Create Type MyInMemoryDataType as Table (<<list the columns>>) as (memory_Optimized=ON)
declare InMemoryTableVariable MyInMemoryDataType
Insert into InMemoryTableVariable
Select  * from SourceDB.dbo.MyDiskBasedTable

InMemory Tables – Hash Index

Couple of quick notes about InMemory Objects:

• Even with “Schema_and_Data” durability option on the Inmemory, the index data is not persisted to the disk. Indexes are rebuilt during the instance startup. This is something to be aware of as it increases the database recovery time.
• All Inmemory Tables need to have at least one index and primary key is required only for “Schema_and_Data” durability, primary key is not necessary for “schema_only” durability option.
• There are only two types of indexes that can be created on InMemory tables – 1. Hash and 2. Range.
• With hash index, hash value of the column is calculated and the memory address for the row is stored in the hash bucket. The same data will have same hash value, however, sometimes two different values can also have the same value.

Hash Match Example:
Create Table InMemoryHash(sno int not null index IX_Sno nonclustered hash with (bucket_count=100))with (Memory_Optimized=ON,Durability=Schema_Only)

Insert into InMemoryHash
Values (1),(2)
--check the maximum chain length
select * from sys.dm_db_xtp_hash_index_stats where object_id=object_id('InMemoryHash')


Insert into InMemoryHash
Values (4),(49)
--check the maximum chain length
select * from sys.dm_db_xtp_hash_index_stats where object_id=object_id('InMemoryHash')


The example shows that 4 and 49 have the same hash value and hence will be stored in the same bucket.  The newer row will be pointed by the hash bucket and the newer row will point to older row for values having same hash match. So, if there are several hash matches, there will be large chain of rows and this can lead to performance issue as optimizer should traverse through all the chains to find the data.

• We need to specify the bucket_count – for hash indexes. The bucket counts are created to the base of two. So, if you specify bucket_count=100 in the table DDL, the actual bucket_count created is 128(2^7). If you specify, bucket_count=1000000, the actual bucket_count created is 1048576(2^20).
• Range Index are implemented similar to regular row index but with some difference. I will cover range indexes  in another post.

Monitoring InMemory Objects and Txns – Memory Utilization

For Inmemory objects, the data exists strictly in the memory. The data may or may not be persisted on to the disk depending on the table durability but that does not change the fact the data always stays in the memory. Monitoring Memory Utilization is always key for database systems but it is more important now than ever because Inmemory transactions cannot continue if memory is not available(no spilling\paging to disk ever happens with InMemory transactions). Also, Inmemory transactions use optimistic concurrency, which is achieved by using multiple row versions and all these stay in the memory as well. With all these complexities around, it is very important to monitor the memory utilization.

Using resource governor, we can create a InMemory resource pool, set up max and min memory limits and bind the Inmemory database to the resource pool, can help limiting the Inmemory objects and transactions memory usage.  By setting up appropriate values, this can help not bringing the entire system down(it still can bring the InMemory transactions down).

The bottom line is with InMemory you always need to monitor the memory utilization.

There are two easy ways to do this:

1.Performance Counters DMV:

select * from sys.dm_os_performance_counters
where counter_name ='XTP Memory Used (KB)' and instance_name = '_Total'

2.Using PerfMon Counters

MSSQL$<<InstanceName>>::Databases --> XTP Memory Used(KB) --> _Total

This gives us the total memory utilization by all InMemory objects and transactions across the instance. You can also track the Inmemory memory usage per individual databases.

 

 

 

 

 

Cannot remove InMemory File Group

To create an InMemory Table – we first have to create a Memory Optimized file group and then associate a “filestream” file type to this file group. After this, InMemory Table can be created.

The catch here(or atleast in SQL 2016 RTM\14 SP1) is you cannot remove the InMemory File or FileGroup even after you dropped all the Inmemory Objects. The only option to remove InMemory file is to script out all the database objects\data and drop the database and recreate with the scripts.

So, if you wanted try InMemory tables and later decide not use them and you can drop the objects but not the InMemory file or file group.

NOTE: InMemory File\File Group can be removed as long as no InMemory Table was ever created. The issue happens only when an InMemory Table was created.

USE [Master]
Go
CREATE DATABASE [InMemoryDB]
CONTAINMENT = NONE
ON PRIMARY
( NAME = N'InMemoryDB', FILENAME = N'C:\Temp\InMemoryDB.mdf' , SIZE = 8192KB , FILEGROWTH = 65536KB ),
FILEGROUP [InMemoryDB] CONTAINS MEMORY_OPTIMIZED_DATA
( NAME = N'InMemoryFile', FILENAME = N'C:\Temp\InMemoryFile' )
LOG ON
( NAME = N'InMemoryDB_log', FILENAME = N'C:\Temp\InMemoryDB_log.ldf' , SIZE = 8192KB , FILEGROWTH = 65536KB )
GO
USE InMemoryDB
Go
Create Table InMemoryTable(Sno int not null primary key nonclustered hash with (bucket_count=1000),sname varchar(20))
with (memory_optimized=ON,Durability=Schema_ONLY)
--Let's drop InMemory Table
Drop Table InMemoryTable
--Trying Removing the InMemory File\FileGroup
ALTER DATABASE [InMemoryDB] REMOVE FILE [InMemoryFile]
ALTER DATABASE [InMemoryDB] REMOVE FILEGROUP [InMemoryDB]
GO
You will get the below error message:

Database Snapshot and InMemory Tables

Database Snapshots are very useful when you need to test some quick changes without having to take backup and, also, they provide way to access the mirrored database to serve READ-ONLY queries.

However, if you have InMemory FileGroup, which is required for InMemory Tables, you cannot use Database Snapshot. I tired this on SQL 2016 RTM version.

USE [Master]
Go
CREATE DATABASE [InMemoryDB]
CONTAINMENT = NONE
ON PRIMARY
( NAME = N'InMemoryDB', FILENAME = N'C:\Temp\InMemoryDB.mdf' , SIZE = 8192KB , FILEGROWTH = 65536KB ),
FILEGROUP [InMemoryDB] CONTAINS MEMORY_OPTIMIZED_DATA
( NAME = N'InMemoryFile', FILENAME = N'C:\Temp\InMemoryFile' )
LOG ON
( NAME = N'InMemoryDB_log', FILENAME = N'C:\Temp\InMemoryDB_log.ldf' , SIZE = 8192KB , FILEGROWTH = 65536KB )
GO
--Creating Snapshot Errors Out
CREATE DATABASE InMemory_Snap ON
( NAME = InMemoryDB, FILENAME =
'C:\Temp\InMemory_Snap.ss' )
AS SNAPSHOT OF InMemoryDB;
GO
Below is the error you will get:

Cannot update primary key in InMemory Tables

InMemory Tables are introduced in SQL 2014 and they are lot improved in SQL 2016.

While the surface area has improved in SQL 2016 compared to 2014 such as Foreign key constraints can be defined between Inmemory Tables, Indexes can have NULL columns, table can be altered etc.

There are still some restrictions in what you can do with SQL 2016 InMemory Tables and one such thing is, modifying the primary key of the table. Typically, we should not be modifying the primary key of the table but if there is ever a need to do that, it cannot be done in SQL 2016 InMemory Tables.

USE [Master]
Go
CREATE DATABASE [InMemoryDB]
CONTAINMENT = NONE
ON PRIMARY
( NAME = N'InMemoryDB', FILENAME = N'C:\Temp\InMemoryDB.mdf' , SIZE = 8192KB , FILEGROWTH = 65536KB ),
FILEGROUP [InMemoryDB] CONTAINS MEMORY_OPTIMIZED_DATA
( NAME = N'InMemoryFile', FILENAME = N'C:\Temp\InMemoryFile' )
LOG ON
( NAME = N'InMemoryDB_log', FILENAME = N'C:\Temp\InMemoryDB_log.ldf' , SIZE = 8192KB , FILEGROWTH = 65536KB )
GO

USE InMemoryDB
Go
Create Table InMemoryTable(Sno int not null primary key nonclustered hash with (bucket_count=1000),sname varchar(20))
with (memory_optimized=ON,Durability=Schema_ONLY)

Insert into InMemoryTable
Values (1,'SomeName')

---Errors out
Update InMemoryTable set sno=2 where Sno=1
You will get error as below: