Showing posts with label SQL. Show all posts
Showing posts with label SQL. Show all posts

Tuesday, June 30, 2015

Bucket List Demo (JavaScript and .Net examples using RESTful API and JSON)

Here are some examples I put together that I really enjoy, because of their simplicity.

The HTML example demonstrates the following:

- MySafeInfo's WICK API (RESTful)
- JavaScript
- jQuery (2.1.4)
- Bootstrap (3.3.5)
- Font Awesome (4.3.0)
- AJAX (asynchronous and synchronous)

 <!DOCTYPE html>  
   
 <html xmlns="http://www.w3.org/1999/xhtml">  
   <head>  
     <title>Bucket List</title>  
   
     <style type="text/css">  
       body {  
         margin: 0px 25px !important;  
       }  
   
       td.icons {  
         font-size: 18px !important;  
       }  
   
       td.icons i {  
         cursor: pointer;  
         padding: 0 15px;  
       }  
   
       .icon-cancel.disabled {  
         color: #cccccc;  
         cursor: not-allowed;  
       }  
     </style>  
   
     <!-- jquery -->  
     <script src="//code.jquery.com/jquery-2.1.4.min.js"></script>  
   
     <!-- bootstrap -->  
     <link href="//maxcdn.bootstrapcdn.com/bootstrap/3.3.5/css/bootstrap.min.css" rel="stylesheet">  
     <script src="//maxcdn.bootstrapcdn.com/bootstrap/3.3.5/js/bootstrap.min.js"></script>  
       
     <!-- font awesome -->  
     <link href="//maxcdn.bootstrapcdn.com/font-awesome/4.3.0/css/font-awesome.min.css" rel="stylesheet" />  
   
     <!-- custom script -->  
     <script type="text/javascript">  
       // variables  
       var readerKey = "28SFkEKyZucdjEvaqYGLD3SY2d9dsnqb";  
       var insertKey = "3InDNvlJnYzSGAHRjT2tUEB4MRygJtuV";  
       var updateKey = "iP3wwSUtuSLFoi4HNzmZIOazxckoldaN";  
       var deleteKey = "LqNQuiJ5Eg8FSzWBJZj2FLO0n0PDrM7L";  
       var table = "BucketList"  
   
       // format  
       String.prototype.format = function () {  
         var args = arguments;  
         return this.replace(/{(\d+)}/g, function (match, number) {  
           return typeof args[number] != 'undefined'  
            ? args[number]  
            : match  
           ;  
         });  
       };  
   
       $(function () {  
         // initialize  
         BindData();  
   
         // bind data  
         function BindData() {  
           // variables  
           var grid = $("#grid");  
   
           // ajax variables  
           var type = "GET";  
           var url = "https://mysafeinfo.com/wick/{0}/tables/{1}".format(readerKey, table);  
           var contentType = "application/json; charset=utf-8";  
           var dataType = "json";  
           var data = "";  
           var async = true;  
   
           // get the data from database table  
           $.ajax({  
             type: type,  
             url: url,  
             contentType: contentType,  
             dataType: dataType,  
             data: data,  
             async: async  
           }).done(function (result) {  
             // variables  
             var result = $(result);  
             var rows = "";  
             var columns = "";  
   
             // heading  
             $("h3.table-results").text("{0} record{1}".format(result.length, (result.length > 1 | result.length == 0) ? "s" : ""));  
   
             // get the template (schema) for this table  
             var template = GetTableTemplate();  
   
             // append a blank record so they can do an inline add in the grid  
             result.splice(0, 0, template);  
   
             // for each row  
             result.each(function (i, item) {  
               // variables  
               var key = "ID";  
               var id = item[key];  
   
               // clear columns  
               columns = "";  
   
               // begin new row  
               rows = "{0}<tr data-id='{1}'>".format(rows, id);  
   
               // get column list  
               for (var column in item) {  
                 // variables  
                 var value = item[column];  
   
                 // get column headers  
                 columns = "{0}<th>{1}</th>".format(columns, column);  
   
                 // check to see if this is an existing record or the blank record for the inline add  
                 // if an existing record show the data as-is  
                 // if the blank record show NEW for the ID column and empty textboxes for the other fields  
                 if (id > 0) {  
                   rows = "{0}<td data-column='{2}' data-value='{1}'>{1}</td>".format(rows, value, column);  
                 }  
                 else {  
                   if (column == key) {  
                     rows = "{0}<td data-column='{2}' data-value='0'><i class='fa fa-plus-circle icon-new' alt='New' title='New'></i></td>".format(rows, value, column);  
                   }  
                   else {  
                     rows = "{0}<td data-column='{2}' data-value='{1}'><input type='text' value='' maxlength='100' data-column='{2}'/></td>".format(rows, value, column);  
                   }  
                 }  
               }  
   
               // check to see if this is an existing record or the blank record for the inline add  
               // if an existing record show the edit and delete icons  
               // if the blank record show the save and cancel icons  
               if (id > 0) {  
                 rows = "{0}<td class='icons text-right'><i class='fa fa-pencil icon-edit' alt='Edit' title='Edit'></i><i class='fa fa-trash-o icon-delete' alt='Delete' title='Delete'></i><i class='fa fa-save icon-save hidden' title='Save' alt='Save'></i><i class='fa fa-times icon-cancel hidden' title='Cancel' alt='Cancel'></i></td></tr>".format(rows);  
               }  
               else {  
                 rows = "{0}<td class='icons text-right'><i class='fa fa-plus icon-save' title='Save' alt='Save'></i><i class='fa fa-times icon-cancel disabled' title='Cancel' alt='Cancel'></i></td></tr>".format(rows);  
               }  
             });  
   
             // columns  
             columns = "{0}<th></th>".format(columns);  
   
             // table  
             var table = "<table id='table-data' class='table table-striped table-responsive table-hover results'><thead><tr>{0}</tr></thead><tbody>{1}</tbody></table>".format(columns, rows);  
   
             // add data to grid  
             grid.html(table);  
           }).always(function () {  
               
           });  
         }  
   
         // edit  
         $("body").delegate("table.results tbody tr td i.icon-edit", "click", function () {  
           // variables  
           var td = $(this).parent("td");  
           var tr = $(td).parent("tr");  
   
           // toggle icons  
           $(td).children("i.icon-edit").addClass("hidden");  
           $(td).children("i.icon-delete").addClass("hidden");  
           $(td).children("i.icon-save").removeClass("hidden");  
           $(td).children("i.icon-cancel").removeClass("hidden");  
   
           // add input fields to each cell  
           for (var i = 1; i < tr.children("td").length - 1; i++) {  
             // variables  
             var cell = tr.children("td")[i];  
   
             // add input to cell  
             $(cell).html("<input type='text' value='{1}' maxlength='100' data-column='{0}'/>".format($(cell).data("column"), $(cell).text().trim()));  
   
             // set focus to the first input  
             if (i == 1) { $(cell).children("input").focus(); }  
           };  
   
           // keydown event to capture enter key  
           $(window).keydown(function (event) {  
             if (event.which == 13) {  
               event.preventDefault();  
               td.children("i.icon-save").trigger("click");  
             }  
           });  
         });  
   
         // delete  
         $("body").delegate("table.results tbody tr td i.icon-delete", "click", function () {  
           if (confirm("Are you sure you want to delete this record?")) {  
             // variables  
             var tr = $(this).parent("td").parent("tr");  
             var id = $(tr).children("td").first().text();  
   
             // ajax variables  
             var type = "DELETE";  
             var url = "https://mysafeinfo.com/wick/{0}/tables/{1}/{2}".format(deleteKey, table, id);  
             var contentType = "application/json; charset=utf-8";  
             var dataType = "json";  
             var data = "";  
             var async = false;  
   
             // delete  
             $.ajax({  
               type: type,  
               url: url,  
               contentType: contentType,  
               dataType: dataType,  
               data: data,  
               async: async  
             }).done(function (result) {  
               // bind data  
               BindData();  
   
               // show message  
               alert(result.Message);  
             }).always(function () {  
                 
             });  
           }  
         });  
   
         // save  
         $("body").delegate("table.results tbody tr td i.icon-save", "click", function () {  
           // variables  
           var tr = $(this).parent("td").parent("tr")  
           var id = tr.data("id");  
           var request = "";  
           var isValid = false;  
   
           // create request  
           for (var i = 1; i < tr.children("td:not(.icons)").length; i++) {  
             // controls  
             var input = $(tr.children("td")[i]).children("input");  
   
             // variables  
             var column = $(input).data("column");  
             var value = $(input).val().trim();  
   
             // append to request  
             request = '{0} "{1}":"{2}",'.format(request, column, value)  
   
             // is valid flag if at least one field has data  
             if (value.length > 0) {  
               isValid = true;  
             }  
           }  
   
           // make sure at least one field has data, otherwise nothing to add/update  
           if (!isValid) {  
             alert("Please enter a value for at least one field.");  
             tr.children("td").find("input").first().focus();  
             tr.children("td").find("input").first().select();  
             return;  
           }  
   
           // format json request  
           request = "{{0}}".format(request.substring(1, request.length - 1));  
   
           // check to see if we need to do an insert or an update  
           if (id > 0) {  
             // ajax variables  
             var type = "PUT";  
             var url = "https://mysafeinfo.com/wick/{0}/tables/{1}/{2}".format(updateKey, table, id);  
             var contentType = "application/json; charset=utf-8";  
             var dataType = "json";  
             var data = request;  
             var async = false;  
   
             // update  
             $.ajax({  
               type: type,  
               url: url,  
               contentType: contentType,  
               dataType: dataType,  
               data: data,  
               async: async  
             }).done(function (result) {  
               // bind  
               BindData();  
   
               // message  
               alert(result.Message);  
             }).always(function () {  
               // remove key event listener  
               RemoveListener();  
             });  
           }  
           else {  
             // ajax variables  
             var type = "POST";  
             var url = "https://mysafeinfo.com/wick/{0}/tables/{1}".format(insertKey, table);  
             var contentType = "application/json; charset=utf-8";  
             var dataType = "json";  
             var data = request;  
             var async = false;  
   
             // insert  
             $.ajax({  
               type: type,  
               url: url,  
               contentType: contentType,  
               dataType: dataType,  
               data: data,  
               async: async  
             }).done(function (result) {  
               // bind  
               BindData();  
   
               // message  
               alert(result.Message);  
             }).always(function () {  
               // remove key event listener  
               RemoveListener();  
             });  
           }  
         });  
   
         // cancel  
         $("body").delegate("table.results tbody tr td i.icon-cancel", "click", function () {  
           // variables  
           var tr = $(this).parent("td").parent("tr")  
           var id = tr.data("id");  
   
           // short-circuit if this is a new record  
           if (id < 1) {  
             return;  
           }  
   
           // set defaults  
           for (var i = 1; i < tr.children("td:not(.icons)").length; i++) {  
             var td = $(tr.children("td")[i]);  
             td.text(td.data("value"));  
           }  
   
           // toggle icons  
           $(this).parent("td").children("i.icon-edit").removeClass("hidden");  
           $(this).parent("td").children("i.icon-delete").removeClass("hidden");  
           $(this).parent("td").children("i.icon-save").addClass("hidden");  
           $(this).parent("td").children("i.icon-cancel").addClass("hidden");  
   
           // remove key event listener  
           RemoveListener();  
         });  
   
         function GetTableTemplate() {  
           // variables  
           var template;  
   
           // ajax variables  
           var type = "GET";  
           var url = "https://mysafeinfo.com/wick/{0}/tables/{1}/template".format(readerKey, table);  
           var contentType = "application/json; charset=utf-8";  
           var dataType = "json";  
           var data = "";  
           var async = false;  
   
           // get table template  
           $.ajax({  
             type: type,  
             url: url,  
             contentType: contentType,  
             dataType: dataType,  
             data: data,  
             async: async  
           }).done(function (result) {  
             template = result;  
           }).always(function () {  
   
           });  
   
           // return  
           return template;  
         }  
   
         function RemoveListener() {  
           $(window).unbind("keydown");  
         }  
       });  
     </script>  
   </head>  
   
   <body>  
     <!-- container for header to indicate how many records were retrieved -->  
     <h3 class="table-results">0 records</h3>  
   
     <!-- container for html table built dynamically from our ajax call -->  
     <div class="table-responsive">  
       <div id="grid"></div>  
     </div>  
   </body>  
 </html>  

I've also included examples using .Net 4.5 (C# and VB.Net) for those looking to call a RESTful API from the server-side, which can be helpful when working with key-based APIs, so you can hide your API keys. The .Net 4.5 examples demonstrate the following:

- MySafeInfo's WICK API (RESTful)
- Json.NET (Newtonsoft.Json)
- WebClient.UploadString

 using System;  
 using System.Net;  
 using Newtonsoft.Json;  
                            
 public class Program  
 {  
      public static void Main()  
      {  
           // variables  
           string Table = "BucketList";  
           string InsertKey = "3InDNvlJnYzSGAHRjT2tUEB4MRygJtuV";  
           string DeleteKey = "LqNQuiJ5Eg8FSzWBJZj2FLO0n0PDrM7L";  
           string InsertUrl = string.Format("https://mysafeinfo.com/wick/{0}/tables/{1}", InsertKey, Table);  
           PostResult PostResult = new PostResult();  
           WickResult WickResult = new WickResult();  
           BucketList BucketList = new BucketList { Description = "Run a mini-marathon", Notes = "2016 OneAmerica 500 Festival Mini-Marathon", Date = "May 2016" };  
   
           // use Json.NET (Newtonsoft.Json) to serialize the data object  
           // https://www.nuget.org/packages/Newtonsoft.Json  
           string Data = JsonConvert.SerializeObject(BucketList);  
   
           // post the data to the MySafeInfo WICK API  
           PostResult = SubmitRequest(Url: InsertUrl, Data: Data);  
   
           // use Json.NET (Newtonsoft.Json) to deserialize json result into a strongly typed data object  
           // https://www.nuget.org/packages/Newtonsoft.Json  
           if (PostResult.Success)  
           {  
                WickResult = JsonConvert.DeserializeObject<WickResult>(PostResult.Message);  
           }  
   
           // debug  
           Console.WriteLine(string.Format("WICK Input: {0}", Data));  
           Console.WriteLine(string.Format("WICK Output: {0}", PostResult.Message));  
           Console.WriteLine(string.Format("ID: {0}", WickResult.ID));  
           Console.WriteLine(string.Format("Success: {0}", WickResult.Success));  
           Console.WriteLine(string.Format("Message: {0}", WickResult.Message));  
   
           // just for fun we are going to delete the record to show how simple a delete is  
           if (WickResult.Success)  
           {  
                Console.WriteLine(string.Format("Delete: {0}", SubmitRequest(Url: string.Format("https://mysafeinfo.com/wick/{0}/tables/{1}/{2}", DeleteKey, Table, WickResult.ID), Method: "DELETE").Message));  
           }  
      }  
   
      public static PostResult SubmitRequest(string Url, string Data = "", string Method = "POST", string ContentType = "application/json; charset=utf-8")  
      {  
           // variables  
           PostResult Result = new PostResult();  
   
           try  
           {  
                // web client  
                using (WebClient Client = new WebClient())  
                {  
                     // content type  
                     Client.Headers[HttpRequestHeader.ContentType] = ContentType;  
   
                     // post data and get result  
                     Result.Message = Client.UploadString(address: Url, method: Method, data: Data);  
                     Result.Success = true;  
                }  
           }  
           catch (Exception ex)  
           {  
                Result.Message = ex.Message;  
           }  
   
           // return  
           return Result;  
      }  
   
      // helper classes  
      public class BucketList  
      {  
           public string Description { get; set; }  
           public string Notes { get; set; }  
           public string Date { get; set; }  
      }  
   
      public class PostResult  
      {  
           public bool Success { get; set; }  
           public string Message { get; set; }  
      }  
   
      public class WickResult  
      {  
           public int ID { get; set; }  
           public bool Success { get; set; }  
           public string Message { get; set; }  
      }  
 }  

As you can see, the server side code is relatively simple since we can leverage Json.NET to serialize and deserialize our data, and we can use built in .Net namespaces (WebClient) to perform our RESTful operations against the API.

Here are links to each example:

Bucket List Demo (HTML, JavaScript, jQuery)
Bucket List Demo (.Net 4.5, C#)
Bucket List Demo (.Net 4.5, VB.Net)

Whether you are using client-side scripting or server side scripting, you can see that using JSON data with a RESTful API can be very easy to implement.

Matt Pavey is a Microsoft Certified software developer who specializes in ASP.Net, VB.Net, C#, AJAX, LINQ, XML, XSL, Web Services, SQL, jQuery, and more. Follow on Twitter @matthewpavey

Sunday, June 14, 2015

SQL Code Generation Tool

The SQL Code Generation Tool is an open source project that I created to help automate the creation of stored procedures, model objects, and the data layer classes in the majority of the projects I'm involved in.

These scripts started in late 2010 when I simply got tired of manually coding VB.Net model classes. Over time, I added additional scripts to code gen the basic stored procedures, model classes, and data layer classes that I generally needed for each entity in my project. Initially the scripts only generated VB.Net code for the model/data layers; however, they now support both VB.Net and C#.

The scripts have went through 3 major rewrites/refactoring to get to their current version (v3.0) and now work with SQLCMD, which makes generating your files (procs, model, data) very simple.

Please keep in mind these scripts were written for a specific style of n-tier architecture, so they may or may not fit your needs out of the box. For most of the applications I work on there is a Web layer, Model layer, Data layer, and a backend SQL Server database (business/data).

These scripts more or less automated 90% of the routine tasks I would encounter after designing a database/table. I almost exclusively use these scripts to create the initial set of stored procedures, model classes (C#, VB.Net), and data layer classes (C#, VB.Net). Once generated I customize them as-needed to fit the project's needs (e.g. adding advanced filtering to stored procedures, adding derived properties to the model class, etc).

Head over to the GitHub repository for more information.

https://github.com/mpavey/sql-code-gen

Matt Pavey is a Microsoft Certified software developer who specializes in ASP.Net, VB.Net, C#, AJAX, LINQ, XML, XSL, Web Services, SQL, jQuery, and more. Follow on Twitter @matthewpavey

Friday, June 12, 2015

Using SQL Server SQLCMD and WinZip wzzip to backup and zip/encrypt a database

Here's an example of how to create a BAT file to backup a SQL Server database using SQLCMD and to zip the backup file using WinZip wzzip with AES256 encryption.
 @ECHO OFF  
   
 set server=(local)  
 set database=Sandbox  
 set bakfile=C:\Users\matt\Desktop\temp\backup.bak  
 set zipfile=C:\Users\matt\Desktop\temp\backup.zip  
 set targetfile=C:\Users\matt\Desktop\temp\backup\backup.zip  
 set password=Password123  
   
 cd C:\Program Files\Microsoft SQL Server\110\Tools\Binn  
 SQLCMD -E -S %server% -Q "BACKUP DATABASE %database% TO DISK = N'%bakfile%' WITH COPY_ONLY, NOFORMAT, INIT"  
   
 IF EXIST "%zipfile%" del /F "%zipfile%"  
   
 cd C:\Program Files\WinZip  
 wzzip "%zipfile%" "%bakfile%" -s%password% -ycAES256  
   
 IF EXIST "%bakfile%" del /F "%bakfile%"  
   
 move "%zipfile%" "%targetfile%"  

You could then use the BAT file to create a scheduled job to automate your backup process.

Matt Pavey is a Microsoft Certified software developer who specializes in ASP.Net, VB.Net, C#, AJAX, LINQ, XML, XSL, Web Services, SQL, jQuery, and more. Follow on Twitter @matthewpavey

Thursday, June 11, 2015

Shredding XML data (elements) using SQL Server

Here's a simple example of shredding XML data (elements) in SQL Server.
 -- variables  
 DECLARE @xml XML = '  
 <dinosaurs>  
  <d>  
   <name>Aachenosaurus</name>  
   <url>http://en.wikipedia.org/wiki/Aachenosaurus</url>  
  </d>  
  <d>  
   <name>Aardonyx</name>  
   <url>http://en.wikipedia.org/wiki/Aardonyx</url>  
  </d>  
  <d>  
   <name>Abdallahsaurus</name>  
   <url>http://en.wikipedia.org/wiki/Abdallahsaurus</url>  
  </d>  
  <d>  
   <name>Abelisaurus</name>  
   <url>http://en.wikipedia.org/wiki/Abelisaurus</url>  
  </d>  
  <d>  
   <name>Abrictosaurus</name>  
   <url>http://en.wikipedia.org/wiki/Abrictosaurus</url>  
  </d>  
  <d>  
   <name>Abrosaurus</name>  
   <url>http://en.wikipedia.org/wiki/Abrosaurus</url>  
  </d>  
  <d>  
   <name>Abydosaurus</name>  
   <url>http://en.wikipedia.org/wiki/Abydosaurus</url>  
  </d>  
  <d>  
   <name>Acanthopholis</name>  
   <url>http://en.wikipedia.org/wiki/Acanthopholis</url>  
  </d>  
  <d>  
   <name>Achelousaurus</name>  
   <url>http://en.wikipedia.org/wiki/Achelousaurus</url>  
  </d>  
  <d>  
   <name>Achillesaurus</name>  
   <url>http://en.wikipedia.org/wiki/Achillesaurus</url>  
  </d>  
 </dinosaurs>'  
   
 -- use common table expression to shred data  
 ;WITH ShreddedData (name, url) AS  
 (  
 SELECT  x.node.value('name[1]', 'VARCHAR(255)'),  
      x.node.value('url[1]', 'VARCHAR(255)')  
 FROM   @xml.nodes('dinosaurs/d') x(node)  
 )  
 SELECT * FROM ShreddedData;  

Shredding XML data (attributes) using SQL Server

Here's a simple example of shredding XML data (attributes) in SQL Server.
 -- variables  
 DECLARE @xml XML = '  
 <dinosaurs>  
  <d name="Aachenosaurus" url="http://en.wikipedia.org/wiki/Aachenosaurus" />  
  <d name="Aardonyx" url="http://en.wikipedia.org/wiki/Aardonyx" />  
  <d name="Abdallahsaurus" url="http://en.wikipedia.org/wiki/Abdallahsaurus" />  
  <d name="Abelisaurus" url="http://en.wikipedia.org/wiki/Abelisaurus" />  
  <d name="Abrictosaurus" url="http://en.wikipedia.org/wiki/Abrictosaurus" />  
  <d name="Abrosaurus" url="http://en.wikipedia.org/wiki/Abrosaurus" />  
  <d name="Abydosaurus" url="http://en.wikipedia.org/wiki/Abydosaurus" />  
  <d name="Acanthopholis" url="http://en.wikipedia.org/wiki/Acanthopholis" />  
  <d name="Achelousaurus" url="http://en.wikipedia.org/wiki/Achelousaurus" />  
  <d name="Achillesaurus" url="http://en.wikipedia.org/wiki/Achillesaurus" />  
 </dinosaurs>'  
   
 -- use common table expression to shred data  
 ;WITH ShreddedData (name, url) AS  
 (  
 SELECT  x.node.value('@name', 'VARCHAR(255)'),  
      x.node.value('@url', 'VARCHAR(255)')  
 FROM   @xml.nodes('dinosaurs/d') x(node)  
 )  
 SELECT * FROM ShreddedData;  

Tuesday, June 9, 2015

Using MVC Web API and SQL Server to create your own PayPal IPN Listener

Recently I was tasked with creating a PayPal IPN Listener, so I started with getting familiar with the PayPal Instant Payment Notification (IPN) documentation:

https://developer.paypal.com/webapps/developer/docs/classic/ipn/integration-guide/IPNIntro/

For this implementation I'm using a MVC Web API Controller (VB.Net) for the listener, along with some helper classes/extensions/methods. I'm also using a SQL Server 2012 table and stored procedure to create a notifications log, which allows you to:

1) Listen
2) Log
3) React

This is important because it allows the listener to do it's job quickly, and lets you focus on the processing later.

Let's start with our SQL Server database Notifications table.
 SET ANSI_NULLS ON  
 GO  
   
 SET QUOTED_IDENTIFIER ON  
 GO  
   
 SET ANSI_PADDING ON  
 GO  
   
 CREATE TABLE [dbo].[Notifications](  
      [NotificationID] [INT] IDENTITY(1,1) NOT NULL,  
      [Timestamp] [DATETIME] NOT NULL CONSTRAINT [DF_Notifications_Timestamp] DEFAULT (GETDATE()),  
      [Type] [VARCHAR](50) NOT NULL CONSTRAINT [DF_Notifications_Type] DEFAULT (''),  
      [IPAddress] [VARCHAR](25) NOT NULL CONSTRAINT [DF_Notifications_IPAddress] DEFAULT (''),  
      [UrlRequest] [VARCHAR](MAX) NOT NULL CONSTRAINT [DF_Notifications_UrlRequest] DEFAULT (''),  
      [UserAgent] [VARCHAR](MAX) NOT NULL CONSTRAINT [DF_Notifications_UserAgent] DEFAULT (''),  
      [Data] [VARCHAR](MAX) NOT NULL CONSTRAINT [DF_Notifications_Message] DEFAULT (''),  
      [Status] [VARCHAR](MAX) NOT NULL CONSTRAINT [DF_Notifications_Status] DEFAULT (''),  
      [Method] [VARCHAR](10) NOT NULL CONSTRAINT [DF_Notifications_Method] DEFAULT (''),  
      [Processed] [BIT] NOT NULL CONSTRAINT [DF_Notifications_Processed] DEFAULT ((0)),  
      [ProcessedDate] [DATETIME] NULL,  
      [Notes] [VARCHAR](MAX) NOT NULL DEFAULT (''),  
      [TransactionID] [VARCHAR](50) NOT NULL DEFAULT (''),  
      [TransactionType] [VARCHAR](50) NOT NULL DEFAULT (''),  
      [ItemName] [VARCHAR](128) NOT NULL DEFAULT (''),  
      [ItemNumber] [VARCHAR](128) NOT NULL DEFAULT (''),  
      [Option] [VARCHAR](200) NOT NULL DEFAULT (''),  
      [Email] [VARCHAR](100) NOT NULL DEFAULT (''),  
      [PaymentStatus] [VARCHAR](25) NOT NULL DEFAULT (''),  
  CONSTRAINT [PK_Notifications] PRIMARY KEY CLUSTERED   
 (  
      [NotificationID] ASC  
 )WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)  
 )  
   
 GO  
   
 SET ANSI_PADDING OFF  
 GO  

Next is the Notifications_Save stored procedure to save data in the Notifications table.
 SET ANSI_NULLS ON  
 GO  
   
 SET QUOTED_IDENTIFIER ON  
 GO  
   
 CREATE PROCEDURE [dbo].[Notifications_Save]  
 (  
      @NotificationID      INT,  
      @Type                VARCHAR(50),  
      @IPAddress           VARCHAR(25),  
      @UrlRequest          VARCHAR(MAX),  
      @UserAgent           VARCHAR(MAX),  
      @Data                VARCHAR(MAX),  
      @Status              VARCHAR(MAX),  
      @Method              VARCHAR(10),  
      @Processed           BIT,  
      @ProcessedDate       DATETIME  
 )  
 AS  
 BEGIN  
      -- check to see if record exists  
      IF EXISTS (SELECT NotificationID FROM dbo.Notifications WHERE NotificationID = @NotificationID)  
      BEGIN  
           -- update  
           UPDATE    dbo.Notifications  
           SET       Type = @Type,  
                     IPAddress = @IPAddress,  
                     UrlRequest = @UrlRequest,  
                     UserAgent = @UserAgent,  
                     Data = @Data,  
                     Status = @Status,  
                     Method = @Method,  
                     Processed = @Processed,  
                     ProcessedDate = @ProcessedDate  
           WHERE     NotificationID = @NotificationID  
      END  
      ELSE  
      BEGIN  
           -- insert  
           INSERT INTO dbo.Notifications  
                (Type, IPAddress, UrlRequest, UserAgent, Data, Status, Method, Processed, ProcessedDate)  
           VALUES  
                (@Type, @IPAddress, @UrlRequest, @UserAgent, @Data, @Status, @Method, @Processed, @ProcessedDate)  
    
           -- get identity value  
           SET @NotificationID = SCOPE_IDENTITY()  
      END  
    
      -- return value  
      RETURN @NotificationID  
 END  

Next we'll create the Notification model object in our MVC project.
 Public Class Notification  
   Public Property NotificationID As Integer = 0  
   Public Property Timestamp As DateTime = DateTime.MinValue  
   Public Property Type As String = String.Empty  
   Public Property IPAddress As String = String.Empty  
   Public Property UrlRequest As String = String.Empty  
   Public Property UserAgent As String = String.Empty  
   Public Property Data As String = String.Empty  
   Public Property Status As String = String.Empty  
   Public Property Method As String = String.Empty  
   Public Property Processed As Boolean = False  
   Public Property ProcessedDate As DateTime?  
 End Class  
   

Then we'll create the Notifications data layer class for calling the stored procedure.
 Imports Microsoft.Practices.EnterpriseLibrary.Data  
 Imports System.Data.Common  
   
 Public Class Notifications     
   Public Function Save(ByVal Notification As Model.Notification) As Integer  
     'variables  
     Dim DB As Database = New DatabaseProviderFactory().CreateDefault()  
     Dim ReturnValue As Integer = -1  
   
     'command  
     Using cmd As DbCommand = DB.GetStoredProcCommand("dbo.Notifications_Save")  
       'parameters  
       With Notification  
         DB.AddInParameter(cmd, "@NotificationID", DbType.Int32, .NotificationID)  
         DB.AddInParameter(cmd, "@Type", DbType.String, .Type)  
         DB.AddInParameter(cmd, "@IPAddress", DbType.String, .IPAddress)  
         DB.AddInParameter(cmd, "@UrlRequest", DbType.String, .UrlRequest)  
         DB.AddInParameter(cmd, "@UserAgent", DbType.String, .UserAgent)  
         DB.AddInParameter(cmd, "@Data", DbType.String, .Data)  
         DB.AddInParameter(cmd, "@Status", DbType.String, .Status)  
         DB.AddInParameter(cmd, "@Method", DbType.String, .Method)  
         DB.AddInParameter(cmd, "@Processed", DbType.Boolean, .Processed)  
         DB.AddInParameter(cmd, "@ProcessedDate", DbType.DateTime, IIf(.ProcessedDate.HasValue AndAlso Not .ProcessedDate.Equals(DateTime.MinValue), .ProcessedDate, DBNull.Value))  
       End With  
   
       'return value parameter  
       DB.AddParameter(cmd, "@ReturnValue", DbType.Int32, 4, ParameterDirection.ReturnValue, False, 0, 0, "@ReturnValue", DataRowVersion.Default, Nothing)  
   
       'execute query  
       DB.ExecuteNonQuery(cmd)  
   
       'get return value  
       ReturnValue = DB.GetParameterValue(cmd, "@ReturnValue")  
     End Using  
   
     'return value  
     Return ReturnValue  
   End Function  
 End Class  

Next we have a few helper classes/extensions/methods that we are going to be calling from the listener controller.

1) IsBlank (extension method)
2) ContainsValue (extension method)
3) Result (model object)
4) PostContentToUrl (shared method)

The IsBlank extension method seems trivial, especially for such a small example, but this is just one of the many extension methods that I keep in my utilities library to keep my code, clean, simple, and consistent.
   <Extension()> _  
   Public Function IsBlank(Value As String) As Boolean  
     Dim ReturnValue As Boolean = True  
   
     If Value IsNot Nothing Then  
       ReturnValue = Value.Trim().Length = 0  
     End If  
   
     Return ReturnValue  
   End Function  

The ContainsValue extension method is another one I keep in my library to to simplify the code so you don't have to deal with case sensitivity all over the place.
   <Extension()> _  
   Public Function ContainsValue(Value As String, CompareValue As String) As Boolean  
     Dim ReturnValue As Boolean = False  
   
     If Value IsNot Nothing AndAlso CompareValue IsNot Nothing Then  
       ReturnValue = Value.Trim().IndexOf(CompareValue.Trim(), StringComparison.OrdinalIgnoreCase) >= 0  
     End If  
   
     Return ReturnValue  
   End Function  

The Result model object is a simple helper class we're going to use when we post our data to the PayPal verification URL, which will allow us to return back a property indicating success or failure, as well as the response body.
 Public Class Result  
   Public Property Success As Boolean = False  
   Public Property Message As String = String.Empty    
 End Class  

Then we have our PostContentToUrl method, which was written with re-usability in mind. It accepts a URL and a request body. This allows you to use it for POST'ing data generically to any service.
   Public Shared Function PostContentToUrl(ByVal Url As String, ByVal Data As String) As Model.Result  
     'variables  
     Dim Result As New Model.Result  
   
     Try  
       'create web request  
       Dim MyRequest As WebRequest = WebRequest.Create(Url)  
       Dim ByteArray As Byte() = Encoding.UTF8.GetBytes(Data)  
   
       'properties  
       MyRequest.Method = "POST"  
       MyRequest.ContentType = "application/x-www-form-urlencoded"  
       MyRequest.ContentLength = ByteArray.Length  
   
       'get the request stream  
       Using DataStream As Stream = MyRequest.GetRequestStream()  
         'write the data to the request stream  
         DataStream.Write(ByteArray, 0, ByteArray.Length)  
   
         'close the stream object  
         DataStream.Close()  
       End Using  
   
       'response  
       Using MyResponse As HttpWebResponse = DirectCast(MyRequest.GetResponse(), HttpWebResponse)  
         Using MyReader As New StreamReader(MyResponse.GetResponseStream())  
           Result.Message = MyReader.ReadToEnd()  
           Result.Success = True  
         End Using  
       End Using  
     Catch ex As Exception  
       Result.Message = ex.Message  
     End Try  
   
     'return  
     Return Result  
   End Function  

Next we'll setup a BaseController class to expose several properties for the current HTTP request.
 Public Class BaseController   
   Inherits System.Web.Http.ApiController   
   
   Public ReadOnly Property QueryString() As String   
    Get   
     Try   
      Return DirectCast(Request.Properties("MS_HttpContext"), HttpContextBase).Request.QueryString.ToString   
     Catch ex As Exception   
      Return String.Empty   
     End Try   
    End Get   
   End Property   
     
   Public ReadOnly Property IPAddress() As String   
    Get   
     Try   
      Return DirectCast(Request.Properties("MS_HttpContext"), HttpContextBase).Request.UserHostAddress   
     Catch ex As Exception   
      Return String.Empty   
     End Try   
    End Get   
   End Property   
     
   Public ReadOnly Property UrlRequest() As String   
    Get   
     Try   
      Return DirectCast(Request.Properties("MS_HttpContext"), HttpContextBase).Request.Url.AbsoluteUri   
     Catch ex As Exception   
      Return String.Empty   
     End Try   
    End Get   
   End Property   
     
   Public ReadOnly Property RawUrl() As String   
    Get   
     Try   
      Return DirectCast(Request.Properties("MS_HttpContext"), HttpContextBase).Request.RawUrl   
     Catch ex As Exception   
      Return String.Empty   
     End Try   
    End Get   
   End Property   
     
   Public ReadOnly Property UrlReferrer() As String   
    Get   
     Try   
      Return DirectCast(Request.Properties("MS_HttpContext"), HttpContextBase).Request.UrlReferrer.AbsoluteUri   
     Catch ex As Exception   
      Return String.Empty   
     End Try   
    End Get   
   End Property   
     
   Public ReadOnly Property UserAgent() As String   
    Get   
     Try   
      Return DirectCast(Request.Properties("MS_HttpContext"), HttpContextBase).Request.UserAgent   
     Catch ex As Exception   
      Return String.Empty   
     End Try   
    End Get   
   End Property   
     
   Public ReadOnly Property Crawler() As Boolean   
    Get   
     Try   
      Return DirectCast(Request.Properties("MS_HttpContext"), HttpContextBase).Request.Browser.Crawler   
     Catch ex As Exception   
      Return False   
     End Try   
    End Get   
   End Property  

   Public ReadOnly Property Method() As String  
     Get  
       Try  
         Return DirectCast(Request.Properties("MS_HttpContext"), HttpContextBase).Request.HttpMethod  
       Catch ex As Exception  
         Return False  
       End Try  
     End Get  
   End Property  
 End Class  

Now that we've got all the core pieces in place let's take a look at the ListenerController, which is the simplest part of the entire process.
 Imports System.Net  
 Imports System.Web.Http  
 Imports System.IO  
   
 Namespace Controllers  
   <RoutePrefix("listener")>  
   Public Class ListenerController  
     Inherits BaseController  
   
     <HttpPost>  
     <Route("ipn")>  
     Public Function InstantPaymentNotification() As String  
       'variables  
       Dim Data As String = String.Empty  
       Dim Notification As New Model.Notification  
   
       'get data  
       Try  
         Data = New StreamReader(HttpContext.Current.Request.InputStream).ReadToEnd()  
       Catch ex As Exception  
         Data = String.Empty  
       End Try  
   
       'check for valid data  
       If Data.IsBlank Then  
         Return String.Empty  
       End If  
   
       'PayPal HTTP POSTs an IPN message to your listener that notifies it of an event.  
       'Your listener returns an empty HTTP 200 response to PayPal.  
       'Your listener HTTP POSTs the complete, unaltered message back to PayPal; the message must contain the same fields (in the same order) as the original message and be encoded in the same way as the original message.  
       'PayPal sends a single word back - either VERIFIED (if the message matches the original) or INVALID (if the message does not match the original).  
   
       'Every IPN message you receive from PayPal includes a User-Agent HTTP request header whose value is PayPal IPN ( https://www.paypal.com/ipn ).  
       'Do not use this header to verify that an IPN really came from PayPal and has not been tampered with.  
       'Rather, to verify these things, you must use the IPN authentication protocol outlined above.  
       If Not MyBase.UserAgent.ContainsValue("PayPal") Then  
         Return String.Empty  
       End If  
   
       'Before you can trust the contents of the message, you must first verify that the message came from PayPal.  
       'To verify the message, you must send back the contents in the exact order they were received and precede it with the command _notify-validate, as follows:  
       Dim VerifyUrl As String = IIf(Data.ContainsValue("test_ipn=1"), "https://www.sandbox.paypal.com/cgi-bin/webscr", "https://www.paypal.com/cgi-bin/webscr")  
       Dim VerifyData As String = String.Format("cmd=_notify-validate&{0}", Data)  
       Dim Result As Model.Result = Functions.PostContentToUrl(Url:=VerifyUrl, Data:=VerifyData)  
   
       'properties  
       Notification.Type = "IPN"  
       Notification.IPAddress = MyBase.IPAddress  
       Notification.UrlRequest = MyBase.UrlRequest  
       Notification.UserAgent = MyBase.UserAgent  
       Notification.Data = Data  
       Notification.Status = Result.Message  
       Notification.Method = MyBase.Method  
   
       'save  
       Using DB As New Data.Notifications  
         DB.Save(Notification)  
       End Using  
   
       'Important: After you have authenticated an IPN message (received a VERIFIED response from PayPal), you must perform these important checks before you can assume that the IPN is both legitimate and has not already been processed:  
       'Check that the payment_status is Completed.  
       'If the payment_status is Completed, check the txn_id against the previous PayPal transaction that you processed to ensure the IPN message is not a duplicate.  
       'Check that the receiver_email is an email address registered in your PayPal account.  
       'Check that the price (carried in mc_gross) and the currency (carried in mc_currency) are correct for the item (carried in item_name or item_number).  
       'Once you have completed these checks, IPN authentication is complete.  
       'Now, you can update your database with the information provided and initiate any back-end processing that's appropriate.  
   
       'return  
       Return String.Empty  
     End Function  
   End Class  
 End Namespace  

Because we've used <RoutePrefix("listener")> for the controller and <Route("ipn")> for the InstantPaymentNotification function, our IPN listener service would be exposed as follows:

http://yourdomain.com/listener/ipn

At this point you have an IPN listener that can:

1) Receive IPN notifications from PayPal
2) Respond back instantly to PayPal for the verification handshake
3) Log notification activity to the [Notifications] table in your database

This is the point in the process where you could go several different directions, and ultimately how you validate and process the data is going to be dependent on what types of transaction types (txn_type) you support and what types of services you are selling (e.g. products, subscriptions, etc).

With that said, here is an example of how you might do some data processing in SQL (e.g. a scheduled job that runs every minute).

We'll start with a Split table-valued function to help with splitting the IPN data vertically.
 SET QUOTED_IDENTIFIER ON  
 GO  

 CREATE FUNCTION [dbo].[Split]  
 (  
      @List          VARCHAR(MAX),  
      @Delimiter     VARCHAR(1)  
 )  
 RETURNS @Table TABLE  
 (   
      ID        INT IDENTITY(1,1),  
      Value     VARCHAR(MAX)  
 )  
 AS  
 BEGIN  
      -- loop through the list  
      WHILE (CHARINDEX(@Delimiter, @List) > 0)  
      BEGIN  
           -- add the value to the table  
           INSERT INTO @Table  
                (Value)  
                SELECT Value = LTRIM(RTRIM(SUBSTRING(@List, 1, CHARINDEX(@Delimiter, @List)-1)))  
   
           -- remove the value from the list  
           SET @List = SUBSTRING(@List, CHARINDEX(@Delimiter, @List) + 1, LEN(@List))  
      END  
   
      -- insert remaining value from the list  
      INSERT INTO @Table  
           (Value)  
           SELECT Value = LTRIM(RTRIM(@List))  
        
      -- return  
      RETURN  
 END  
   
 GO  

Next we'll use a SQL script to process any unprocessed notifications.
 /*  
 Important: After you have authenticated an IPN message (received a VERIFIED response from PayPal), you must perform these important checks before you can assume that the IPN is both legitimate and has not already been processed:  
 Check that the payment_status is Completed.  
 If the payment_status is Completed, check the txn_id against the previous PayPal transaction that you processed to ensure the IPN message is not a duplicate.  
 Check that the receiver_email is an email address registered in your PayPal account.  
 Check that the price (carried in mc_gross) and the currency (carried in mc_currency) are correct for the item (carried in item_name or item_number).  
 Once you have completed these checks, IPN authentication is complete.  
 Now, you can update your database with the information provided and initiate any back-end processing that's appropriate.  
 */  
   
 -- set nocount on  
 SET NOCOUNT ON;  
   
 -- if status is not verified mark it as processed and move on  
 UPDATE    dbo.Notifications  
 SET       Processed = 1,  
           ProcessedDate = GETDATE(),  
           Notes = 'Invalid'  
 WHERE     Type = 'IPN'  
 AND       Processed = 0  
 AND       Status != 'VERIFIED'  
   
 -- temp table to get a list of records that are being processed in this batch
 DECLARE @Batch TABLE  
 (  
      NotificationID     INT              NOT NULL DEFAULT 0  
 )  
   
 -- temp table to split out the IPN data vertically
 DECLARE @Details TABLE  
 (  
      BatchID            VARCHAR(50)      NOT NULL DEFAULT '',  
      NotificationID     INT              NOT NULL DEFAULT 0,  
      Property           VARCHAR(50)      NOT NULL DEFAULT '',  
      Value              VARCHAR(MAX)     NOT NULL DEFAULT ''  
 )  
   
 -- get unprocessed notifications that are verified and save in temp table  
 INSERT INTO @Batch  
   (NotificationID)  
      SELECT       NotificationID  
      FROM         dbo.Notifications  
      WHERE        Type = 'IPN'  
      AND          Processed = 0  
      AND          Status = 'VERIFIED'  
      ORDER BY     NotificationID  
   
 -- declare variables for cursor  
 DECLARE @NotificationID    INT           = 0  
 DECLARE @Data              VARCHAR(MAX)  = ''  
   
 -- declare cursor  
 DECLARE MyCursor CURSOR LOCAL FAST_FORWARD  
 FOR  
      SELECT       n.NotificationID,  
                   n.Data  
      FROM         dbo.Notifications n  
      JOIN         @Batch x ON n.NotificationID = x.NotificationID  
      ORDER BY     n.NotificationID  
   
 -- open cursor  
 OPEN MyCursor  
   
 -- get the first result  
 FETCH NEXT FROM MyCursor INTO @NotificationID, @Data  
   
 -- loop through results  
 WHILE @@FETCH_STATUS = 0  
 BEGIN       
      -- variables  
      DECLARE @BatchID    VARCHAR(50) = FORMAT(GETDATE(), 'yyyyMMddHHmmssfffffff') + dbo.GetRandomString(5)  
        
      -- split IPN data vertically
      INSERT INTO @Details  
           (BatchID, NotificationID, Property, Value)  
           SELECT    BatchID = @BatchID,  
                     @NotificationID,  
                     Property = (SELECT Value FROM dbo.Split(Value, '=') WHERE ID = 1),  
                     Value = (SELECT dbo.DecodeValue(Value) FROM dbo.Split(Value, '=') WHERE ID = 2)  
           FROM      dbo.Split(@Data, '&')  
   
      -- fetch the next record  
      FETCH NEXT FROM MyCursor INTO @NotificationID, @Data  
 END  
   
 -- close cursor  
 CLOSE          MyCursor  
 DEALLOCATE     MyCursor  
   
 -- extract core information that we are interested in for our payment processing
 -- txn_id, txn_type, item_name, item_number, option_selection1, custom, payment_status
 UPDATE    n  
 SET       n.TransactionID = ISNULL(x.txn_id, ''),  
           n.TransactionType = ISNULL(x.txn_type, ''),  
           n.ItemName = ISNULL(x.item_name, ''),  
           n.ItemNumber = ISNULL(x.item_number, ''),  
           n.[Option] = ISNULL(x.option_selection1, ''),  
           n.Email = ISNULL(x.custom, ''),  
           n.PaymentStatus = ISNULL(x.payment_status, '')  
 FROM      @Details d  
 PIVOT     (  
                MAX(d.Value)  
                FOR Property IN (txn_id, txn_type, item_name, item_number, option_selection1, custom, payment_status)  
           )     AS x  
 JOIN      dbo.Notifications n ON x.NotificationID = n.NotificationID  
   
 -- process non-payments  
 UPDATE    n  
 SET       n.Processed = 1,  
           n.ProcessedDate = GETDATE(),  
           n.Notes = 'Non-payment'  
 FROM      dbo.Notifications n  
 JOIN      @Batch x ON n.NotificationID = x.NotificationID  
 WHERE     n.Processed = 0  
 AND       (LEN(n.TransactionID) = 0 OR LEN(n.PaymentStatus) = 0)  
   
 -- process duplicate transactions  
 UPDATE    n  
 SET       n.Processed = 1,  
           n.ProcessedDate = GETDATE(),  
           n.Notes = 'Duplicate'  
 FROM      dbo.Notifications n  
 JOIN      @Batch x ON n.NotificationID = x.NotificationID  
 WHERE     n.Processed = 0  
 AND       n.TransactionID IN (SELECT y.TransactionID FROM dbo.Notifications y WHERE y.Processed = 1)  
   
 -- process non-completed payments  
 UPDATE    n  
 SET       n.Processed = 1,  
           n.ProcessedDate = GETDATE(),  
           n.Notes = 'Unknown payment status'  
 FROM      dbo.Notifications n  
 JOIN      @Batch x ON n.NotificationID = x.NotificationID  
 WHERE     n.Processed = 0  
 AND       n.PaymentStatus != 'Completed'  
   
 -- process completed payments  
 UPDATE    n  
 SET       n.Processed = 1,  
           n.ProcessedDate = GETDATE(),  
           n.Notes = 'Completed'  
 FROM      dbo.Notifications n  
 JOIN      @Batch x ON n.NotificationID = x.NotificationID  
 WHERE     n.Processed = 0  
 AND       n.PaymentStatus = 'Completed'  

This is the point where if you have new "completed" payments you could trigger other workflow items in your system, such as email notifications to send a license key, or renewing a subscription, or otherwise giving access to the product/service to the payee.

Remember this is just a starting point and your specific data processing implementation will vary based on your needs.

Matt Pavey is a Microsoft Certified software developer who specializes in ASP.Net, VB.Net, C#, AJAX, LINQ, XML, XSL, Web Services, SQL, jQuery, and more. Follow on Twitter @matthewpavey

Sunday, May 3, 2015

ASP.Net C# SQL Building Your Own Url Shortner

Several years ago I had to build a custom URL shortner for a website. I recently had a task where I needed to do something similar, so I converted it to C# and thought I'd share how easy this can be.

First off you'll need a database table for the url data. In this example I'm using SQL Server 2008.
 SET ANSI_NULLS ON  
 GO  
 SET QUOTED_IDENTIFIER ON  
 GO  
 SET ANSI_PADDING ON  
 GO  
 CREATE TABLE [dbo].[Urls](  
      [PK] [int] IDENTITY(1,1) NOT NULL,  
      [Key] [varchar](10) COLLATE SQL_Latin1_General_CP1_CS_AS NOT NULL CONSTRAINT [DF_Urls_Key] DEFAULT (''),  
      [Url] [varchar](2000) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL CONSTRAINT [DF_Urls_Url] DEFAULT (''),  
      [CreatedBy] [varchar](100) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL,  
      [CreatedDate] [smalldatetime] NOT NULL CONSTRAINT [DF_Urls_CreatedDate] DEFAULT (getdate()),  
  CONSTRAINT [PK_Urls] PRIMARY KEY CLUSTERED   
 (  
      [PK] ASC  
 )WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY],  
  CONSTRAINT [IX_Urls] UNIQUE NONCLUSTERED   
 (  
      [Key] ASC  
 )WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]  
 ) ON [PRIMARY]  
   
 GO  
 SET ANSI_PADDING OFF  
 GO  

PK column to act as the primary key
- Key column is a unique nonclustered column, which is the unique key for the shortened url
- Url column which is the fully qualified url that that the shortened url should redirect to
- CreatedBy is a simple varchar field to provide optional auditing
CreatedDate is a simple smalldatetime field to provide optional auditing

It's worth noting that I'm using SQL_Latin1_General_CP1_CS_AS for the collation on the Key column. This isn't absolutely necessary; however, it means that the Key column is case-sensitive and makes the universe much larger for the number of unique values that be be stored in that column.

The next step is to create the view and scalar-valued functions.

GetUniqueIdentifierView
 SET ANSI_NULLS ON  
 GO  
 SET QUOTED_IDENTIFIER ON  
 GO  
 CREATE VIEW [dbo].[GetUniqueIdentifierView]  
 AS  
 SELECT 'ID' = NEWID()  
 GO  

GetUniqueIdentifier
 SET ANSI_NULLS ON  
 GO  
 SET QUOTED_IDENTIFIER ON  
 GO  
 CREATE FUNCTION [dbo].[GetUniqueIdentifier]()  
 RETURNS UNIQUEIDENTIFIER  
 AS   
 BEGIN  
      RETURN (SELECT ID FROM GetUniqueIdentifierView)  
 END  
 GO  

GetRandomString
 SET ANSI_NULLS OFF  
 GO  
 SET QUOTED_IDENTIFIER ON  
 GO  
 CREATE FUNCTION [dbo].[GetRandomString]  
 (  
      @Length     INT = 5  
 )  
 RETURNS VARCHAR(MAX)  
 AS  
 BEGIN  
      -- variables  
      DECLARE @Characters     VARCHAR(62)  
      DECLARE @Output         VARCHAR(MAX)  
      DECLARE @i              INT  
        
      -- values  
      SET @Characters = '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz'  
      SET @Output = ''  
      SET @i = 0  
        
      -- loop  
      WHILE @i < @Length  
      BEGIN  
           -- variables  
           DECLARE @Position INT  
             
           -- random position in character map  
           SET @Position = ABS(CAST(CAST(dbo.GetUniqueIdentifier() AS VARBINARY) AS INT)) % LEN(@Characters)  
             
           -- concatenate the random character map value to our string  
           SET @Output = @Output + SUBSTRING(@Characters, @Position + 1, 1)  
   
           -- increment loop counter       
           SET @i = @i + 1  
      END  
        
      -- return value  
      RETURN ISNULL(@Output, '')  
 END  
 GO  

At this point we have a simple function we can call to get a random string of a specified length, for example:
 SELECT dbo.GetRandomString(5)  

Now we'll create the stored procedures to finish off the SQL side of things.

The first stored procedure is Urls_List. This stored procedure has optional parameters to either return all urls, return a single url for the specified PK, or return a single url for the specified Key.
 SET ANSI_NULLS OFF  
 GO  
 SET QUOTED_IDENTIFIER ON  
 GO  
 CREATE PROCEDURE [dbo].[Urls_List]  
 (  
      @PK      INT             = 0,  
      @Key     VARCHAR(10)     = ''  
 )  
 AS  
 BEGIN  
      -- SET NOCOUNT ON added to prevent extra result sets from interfering with SELECT statements.  
      SET NOCOUNT ON;  
   
      -- get data  
      SELECT        PK,  
                    [Key],  
                    Url,  
                    'UrlShort' = '{BaseUrl}url/?' + [Key],  
                    CreatedBy,  
                    CreatedDate  
      FROM          Urls  
      WHERE         PK =   
                          CASE  
                               WHEN @PK > 0 THEN @PK  
                               ELSE PK  
                          END  
      AND               [Key] =   
                          CASE  
                               WHEN LEN(@Key) > 0 THEN @Key  
                               ELSE [Key]  
                          END  
      ORDER BY      PK  
 END  
 GO  

Now you have a simple way to get the data from the Urls table, for example:
 -- get all urls  
 EXEC dbo.Urls_List  
   
 -- get url for specified PK  
 EXEC dbo.Urls_List @PK = 1  
   
 -- get url for specified Key  
 EXEC dbo.Urls_List @Key = 'gyzAq'  

You'll want to pay special attention to the {BaseUrl} reference in the Urls_List stored procedure. You could do one of two things here. You could plug your fully qualified base url directly in there (e.g. http://pavey.me/) OR you can keep that placeholder in there, and replace it generically in either the data layer, model object, or the web project, depending on your requirements.

The next stored procedure is Urls_Save. This stored procedure creates the record in the Urls table with a unique key and returns the record. If a record with the specified Url already exists it will return that record instead of creating a new one.
 SET ANSI_NULLS OFF  
 GO  
 SET QUOTED_IDENTIFIER ON  
 GO  
 CREATE PROCEDURE [dbo].[Urls_Save]  
 (  
      @Url          VARCHAR(2000),  
      @CreatedBy    VARCHAR(100)  
 )  
 AS  
 BEGIN  
      -- declare variables  
      DECLARE @PK     INT  
        
      -- see if url already exists  
      SELECT   @PK = PK  
      FROM     Urls  
      WHERE    Url = @Url  
        
      -- handle nulls  
      SET @PK = ISNULL(@PK, 0)  
        
      -- create url if necessary  
      IF @PK < 1  
      BEGIN  
           -- variables  
           DECLARE @Key            VARCHAR(5) = ''  
           DECLARE @KeyLength      INT        = 5  
           DECLARE @KeyIsUnique    BIT        = 0  
             
           -- get random key  
           SET @Key = dbo.GetRandomString(@KeyLength)  
   
           -- check to see if key already exists  
           IF NOT EXISTS(SELECT PK FROM Urls WHERE [Key] = @Key)  
           BEGIN  
                SET @KeyIsUnique = 1  
           END  
   
           -- if key is not unique keep looking  
           IF @KeyIsUnique = 0  
           BEGIN  
                WHILE @KeyIsUnique = 0  
                BEGIN  
                     -- get random key  
                     SET @Key = dbo.GetRandomString(@KeyLength)  
   
                     -- check to see if key already exists  
                     IF NOT EXISTS(SELECT PK FROM Urls WHERE [Key] = @Key)  
                     BEGIN  
                          SET @KeyIsUnique = 1  
                     END  
                END  
           END  
   
           -- insert record  
           INSERT INTO Urls  
                ([Key], Url, CreatedBy)  
           VALUES  
                (@Key, @Url, @CreatedBy)  
                  
           -- get identity  
           SET @PK = SCOPE_IDENTITY()  
      END  
   
      -- return url  
      EXEC Urls_List @PK = @PK  
 END  
 GO  

You'll notice in the Urls_Save stored procedure I'm using @KeyLength = 5. The GetRandomString function we created can handle creating a random string of any size, so you'll just want to set this based on your needs. In my case, a unique key length of 5 was sufficient.

This takes care of the SQL side of things. Now we are going to create a model class to represent our url data. I keep my model classes in a Model project, and I use a BaseModel class, which for purposes of this example is empty, but gives you a place to provide base support for your model classes.

BaseModel.cs
 using System;  
 using System.Collections.Generic;  
 using System.Linq;  
 using System.Text;  
   
 namespace Sandbox.Model  
 {  
   public class BaseModel : IDisposable  
   {  
   
     #region "IDisposable Support"  
     // To detect redundant calls  
     private bool disposedValue;  
   
     // IDisposable  
     protected virtual void Dispose(bool disposing)  
     {  
       if (!this.disposedValue)  
       {  
         if (disposing)  
         {  
           // TODO: dispose managed state (managed objects).  
         }  
   
         // TODO: free unmanaged resources (unmanaged objects) and override Finalize() below.  
         // TODO: set large fields to null.  
       }  
       this.disposedValue = true;  
     }  
   
     // TODO: override Finalize() only if Dispose(ByVal disposing As Boolean) above has code to free unmanaged resources.  
     //Protected Overrides Sub Finalize()  
     //  ' Do not change this code. Put cleanup code in Dispose(ByVal disposing As Boolean) above.  
     //  Dispose(False)  
     //  MyBase.Finalize()  
     //End Sub  
   
     // This code added by Visual Basic to correctly implement the disposable pattern.  
     public void Dispose()  
     {  
       // Do not change this code. Put cleanup code in Dispose(ByVal disposing As Boolean) above.  
       Dispose(true);  
       GC.SuppressFinalize(this);  
     }  
     #endregion  
   }  
 }  

UrlShort.cs
 using System;  
 using System.Collections.Generic;  
 using System.Linq;  
 using System.Text;  
   
 namespace Sandbox.Model  
 {  
   public class UrlShortner : BaseModel  
   {  
     // constructor  
     public UrlShortner()  
     {  
       PK = 0;  
       Key = string.Empty;  
       Url = string.Empty;  
       UrlShort = string.Empty;  
       CreatedBy = string.Empty;  
       CreatedDate = DateTime.MinValue;  
     }  
   
     // public properties  
     public int PK { get; set; }  
     public string Key { get; set; }  
     public string Url { get; set; }  
     public string UrlShort { get; set; }  
     public string CreatedBy { get; set; }  
     public DateTime CreatedDate { get; set; }  
   }  
 }  

Now we'll create the data layer class. I keep the data layer classes in a Data project, and I use a BaseDB class, which for purposes of this example is empty, but gives you a place to provide base support for your data layer classes. In this example we're using Microsoft Enterprise Library 5.0 – May 2011 to provide the data access.

BaseDB.cs
 using System;  
 using System.Collections.Generic;  
 using System.Linq;  
 using System.Text;  
   
 namespace Sandbox.Data  
 {  
   public class BaseDB : IDisposable  
   {  
   
     #region "IDisposable Support"  
     // To detect redundant calls  
     private bool disposedValue;  
   
     // IDisposable  
     protected virtual void Dispose(bool disposing)  
     {  
       if (!this.disposedValue)  
       {  
         if (disposing)  
         {  
           // TODO: dispose managed state (managed objects).  
         }  
   
         // TODO: free unmanaged resources (unmanaged objects) and override Finalize() below.  
         // TODO: set large fields to null.  
       }  
       this.disposedValue = true;  
     }  
   
     // TODO: override Finalize() only if Dispose(ByVal disposing As Boolean) above has code to free unmanaged resources.  
     //Protected Overrides Sub Finalize()  
     //  ' Do not change this code. Put cleanup code in Dispose(ByVal disposing As Boolean) above.  
     //  Dispose(False)  
     //  MyBase.Finalize()  
     //End Sub  
   
     // This code added by Visual Basic to correctly implement the disposable pattern.  
     public void Dispose()  
     {  
       // Do not change this code. Put cleanup code in Dispose(ByVal disposing As Boolean) above.  
       Dispose(true);  
       GC.SuppressFinalize(this);  
     }  
     #endregion  
   }  
 }  
   

Urls.cs
 using System;  
 using System.Collections.Generic;  
 using System.Linq;  
 using System.Text;  
 using System.Data;  
 using System.Data.Common;  
 using Microsoft.Practices.EnterpriseLibrary.Data;  
 using Sandbox.Utilities;  
   
 namespace Sandbox.Data  
 {  
   public class Urls : BaseDB  
   {  
     public Model.UrlShortner Get(int PK)  
     {  
       // validate  
       if (PK < 1)  
       {  
         return new Model.UrlShortner();  
       }  
   
       // lookup  
       List<Model.UrlShortner> x = List(PK: PK);  
   
       // return  
       if (x.Count > 0)  
       {  
         return x.First();  
       }  
       else  
       {  
         return new Model.UrlShortner();  
       }  
     }  
   
     public Model.UrlShortner Get(string Key)  
     {  
       // validate  
       if (Key.IsBlank())  
       {  
         return new Model.UrlShortner();  
       }  
   
       // lookup  
       List<Model.UrlShortner> x = List(Key: Key);  
   
       // return  
       if (x.Count > 0)  
       {  
         return x.First();  
       }  
       else  
       {  
         return new Model.UrlShortner();  
       }  
     }  
   
     public List<Model.UrlShortner> List(Int32 PK = 0, string Key = "")  
     {  
       // variables  
       Database DB = DatabaseFactory.CreateDatabase();  
       List<Model.UrlShortner> Urls = new List<Model.UrlShortner>();  
   
       // command  
       using (DbCommand cmd = DB.GetStoredProcCommand("dbo.Urls_List"))  
       {  
         // parameters  
         DB.AddInParameter(cmd, "@PK", DbType.Int32, PK);  
         DB.AddInParameter(cmd, "@Key", DbType.String, Key);  
   
         // execute query and get results  
         using (DataTable DT = new DataTable())  
         {  
           using (IDataReader IDR = DB.ExecuteReader(cmd))  
           {  
             if (IDR != null)  
             {  
               DT.Load(IDR);  
             }  
           }  
   
           // convert to business object  
           Urls = DT.ToList<Model.UrlShortner>().ToList();  
         }  
       }  
   
       // return list  
       return Urls;  
     }  
   
     public Model.UrlShortner Save(string Url, string CreatedBy = "")  
     {  
       // variables  
       Database DB = DatabaseFactory.CreateDatabase();  
       Model.UrlShortner UrlShort = new Model.UrlShortner();  
   
       // command  
       using (DbCommand cmd = DB.GetStoredProcCommand("dbo.Urls_Save"))  
       {  
         // parameters  
         DB.AddInParameter(cmd, "@Url", DbType.String, Url);  
         DB.AddInParameter(cmd, "@CreatedBy", DbType.String, CreatedBy);  
   
         // execute query and get results  
         using (DataTable DT = new DataTable())  
         {  
           using (IDataReader IDR = DB.ExecuteReader(cmd))  
           {  
             if (IDR != null)  
             {  
               DT.Load(IDR);  
             }  
           }  
   
           // convert to business object  
           if (DT.Rows.Count > 0)  
           {  
             UrlShort = DT.Rows[0].ToObject<Model.UrlShortner>();  
           }            
         }  
       }  
   
       // return value  
       return UrlShort;  
     }  
   }  
 }  

It might seem like overkill for this example, but I'm using some extension methods to make the data layer much more generic. For example, instead of having to do manual column mapping to map the data from the DataTable to the model object I am able to use ToObject and ToList to do that generically. You wouldn't believe how much time generic extension methods like this will save you over the course of your project.

Extensions.cs
 using System;  
 using System.Collections.Generic;  
 using System.Linq;  
 using System.Reflection;  
 using System.Data;  
 using System.ComponentModel;
   
 namespace Sandbox.Utilities  
 {  
   public static class Extensions  
   {  
     private static Dictionary<Type, IList<PropertyInfo>> typeDictionary = new Dictionary<Type, IList<PropertyInfo>>();  
   
     public static IList<PropertyInfo> GetPropertiesForType<T>()  
     {  
       // variables  
       var type = typeof(T);  
   
       // get types  
       if (!typeDictionary.ContainsKey(typeof(T)))  
       {  
         typeDictionary.Add(type, type.GetProperties().ToList());  
       }  
   
       // return  
       return typeDictionary[type];  
     }  
   
     public static T ToObject<T>(this DataRow row) where T : new()  
     {  
       // variables  
       IList<PropertyInfo> properties = GetPropertiesForType<T>();  
   
       // return  
       return CreateItemFromRow<T>(row, properties);  
     }  
   
     public static IList<T> ToList<T>(this DataTable table) where T : new()  
     {  
       // variables  
       IList<T> result = new List<T>();  
   
       // foreach  
       foreach (DataRow row in table.Rows)  
       {  
         result.Add(row.ToObject<T>());  
       }  
   
       // return  
       return result;  
     }  
   
     private static T CreateItemFromRow<T>(DataRow row, IList<PropertyInfo> properties) where T : new()  
     {  
       // variables  
       T item = new T();  
   
       // foreach  
       foreach (var property in properties)  
       {  
         // make sure a column exists in the table with this property name  
         if (row.Table.Columns.Contains(property.Name))  
         {  
           // get the value from the current data row  
           object value = row[property.Name];  
   
           // set property accordingly  
           if (value != null & value != DBNull.Value)  
           {  
             SetProperty<T>(item, property.Name, value);  
           }  
         }  
       }  
   
       // return  
       return item;  
     }  
   
     public static string GetProperty<T>(this T obj, string Property)  
     {  
       // reflection  
       PropertyInfo propertyInfo = obj.GetType().GetProperty(Property, BindingFlags.Public | BindingFlags.Instance | BindingFlags.IgnoreCase);  
       object property = null;  
   
       // make sure property is valid  
       if (propertyInfo != null)  
       {  
         property = propertyInfo.GetValue(obj, null);  
       }  
   
       // return value  
       if (property != null)  
       {  
         return property.ToString();  
       }  
       else  
       {  
         return string.Empty;  
       }  
     }  
   
     public static T SetProperty<T>(this T obj, string Property, object Value)  
     {  
       // reflection  
       PropertyInfo prop = obj.GetType().GetProperty(Property, BindingFlags.Public | BindingFlags.Instance | BindingFlags.IgnoreCase);  
   
       // trim strings  
       if (Value.GetType() == typeof(string))  
       {  
         Value = Value.ToString().Trim();  
       }  
   
       // make sure property is valid  
       if (prop != null && prop.CanWrite)  
       {  
         prop.SetValue(obj, Value, null);  
       }  
   
       // return  
       return obj;  
     }  

     public static bool HasValue(this string Value)  
     {  
       return !Value.IsBlank();  
     }  
   
     public static bool IsBlank(this string Value)  
     {  
       bool ReturnValue = true;  
   
       if (Value != null)  
       {  
         ReturnValue = Value.Trim().Length == 0;  
       }  
   
       return ReturnValue;  
     }  
   }  
 }  

Now to round this all out we need to create a shortened url and use it. To create a shortened url you simply call the Save method from the data layer:
 // create shorterned url  
 using (Data.Urls DB = new Data.Urls())  
 {  
    using (Model.UrlShortner MyUrl = DB.Save(Url: "http://www.google.com", CreatedBy: "Testing"))  
    {  
       // MyUrl now has a reference to your shortened url
    }  
 }  

I typically would use a shortened url like this if I was going to include some kind of custom url for a user, for example resetting their password. The url may include an encrypted token and could look pretty ugly, so having a way to provide a nice clean url makes that much more appealing for the email.

Example Url:
http://dev.sandbox.com/forgot-password.aspx?q=urXsbqy9knIpqqOddkFICXTAspidUXCmPS2vJsLqkVanePU%2bGgxLahBGGl1EA%2f%2fXyXxGT4EX6kcqYSvy8BTpib6eEB61Q5WxNcNfEjk1OYx4HGyvO5oKs34JZ%2f1p9jMuAvVpkbkKpBBjYD2UiotIiYob%2baSzHmxRUuUJYkRepd6kSosnXOXssKVQJ%2bWbQQkYfNZWt2OUe9nGw1UOBd1aeO3O9JyEqSiMkYoIF1blW3f9MUx461LSnB9FL2Q4Vbn%2bDGyK0kEHPQVaz5fhpSXIPrfN3Q%2flRtjyqWDXrRnx2%2bE%3e

Example Shortened Url:
http://dev.sandbox.com/url/?Z4cgw

So now we can create the shortened url and know what it looks like, all that's left to do is allow your web application to actually recognize the shortened url and redirect to the actual url.

Start off by creating a new folder in your web project. I called mine url, which is why in the example shortened url above I used /url. This could literally be any folder name you wanted, but something short makes sense.

After you've got the folder created, create a Default.aspx page in that folder, with the following:

Default.aspx
 <%@ Page Language="C#" AutoEventWireup="true" CodeBehind="Default.aspx.cs" Inherits="Sandbox.Web.url.Default" %>  
   
 <!-- see code behind -->  

Default.aspx.cs
 using System;  
 using System.Collections.Generic;  
 using System.Linq;  
 using System.Web;  
 using System.Web.UI;  
 using System.Web.UI.WebControls;  
 using Sandbox.Utilities;  
   
 namespace Sandbox.Web.url  
 {  
   public partial class Default : System.Web.UI.Page  
   {  
     protected void Page_Load(object sender, EventArgs e)  
     {  
       if (!Page.IsPostBack)  
       {  
         ProcessUrl();  
       }  
     }  
   
     private void ProcessUrl()  
     {  
       // variables  
       string Key = Request.QueryString.ToString();  
       string Url = string.Empty;  
   
       // check database for url  
       if (Key.HasValue())  
       {  
         using (Data.Urls DB = new Data.Urls())  
         {  
           using (Model.UrlShortner x = DB.Get(Key: Key))  
           {  
             Url = x.Url;  
           }  
         }  
       }  
   
       // redirect  
       if (Url.HasValue())  
       {  
         Response.Redirect(Url);  
         Response.End();  
       }  
       else  
       {  
         Response.Redirect("~/home.aspx");  
         Response.End();  
       }  
     }  
   }  
 }  

This page simply takes the querystring value and checks to see if there is a matching record in the Urls table with the specified Key. If found, it redirects to that url. If not, it falls back to redirecting the user to some default/home page.

This is just one way to build a URL shortner. You can strip out the requirement for the extension methods or change it to use your own style of model/data classes, but the overall concept doesn't change:

- Database table for the url data
- Views/Functions/Procs for creating/accessing the url data
- Model object for representing the url data
- Data layer class for calling the stored procedures and returning the model object
- Folder for processing the shortened url and redirecting appropriately

Matt Pavey is a Microsoft Certified software developer who specializes in ASP.Net, VB.Net, C#, AJAX, LINQ, XML, XSL, Web Services, SQL, jQuery, and more. Follow on Twitter @matthewpavey

Monday, April 27, 2015

SQL dm_exec_describe_first_result_set dynamic management function

Here's how to call the dm_exec_describe_first_result_set dynamic management function.

This works for SQL 2012 through current version.

SELECT  *
FROM  sys.dm_exec_describe_first_result_set('StoredProcedureName', NULL, 1)

This dynamic management function takes a Transact-SQL statement as a parameter and describes the metadata of the first result set for the statement.

This is a really useful function if you ever need to dynamically figure out the metadata that will be returned for a given stored procedure. For example, if you just need to figure out the column name and data type of each column in the result set, you can do the following:

SELECT  'Property' = name,
'SqlType' = system_type_name
FROM sys.dm_exec_describe_first_result_set('StoredProcedureName', NULL, 1)

For specific syntax, arguments, and what data is returned visit MSDN.

Matt Pavey is a Microsoft Certified software developer who specializes in ASP.Net, VB.Net, C#, AJAX, LINQ, XML, XSL, Web Services, SQL, jQuery, and more. Follow on Twitter @matthewpavey

Thursday, February 10, 2011

SQL Server Reorganizing and Rebuilding Indexes

"The SQL Server Database Engine automatically maintains indexes whenever insert, update, or delete operations are made to the underlying data. Over time these modifications can cause the information in the index to become scattered in the database (fragmented). Fragmentation exists when indexes have pages in which the logical ordering, based on the key value, does not match the physical ordering inside the data file. Heavily fragmented indexes can degrade query performance and cause your application to respond slowly."

"You can remedy index fragmentation by either reorganizing an index or by rebuilding an index. For partitioned indexes built on a partition scheme, you can use either of these methods on a complete index or on a single partition of an index."

http://technet.microsoft.com/en-us/library/ms187874.aspx

http://technet.microsoft.com/en-us/library/ms189858.aspx

http://technet.microsoft.com/en-us/library/ms188388.aspx

Tuesday, November 9, 2010

Nested FOR XML results with SQL Server’s PATH mode

Query output as XML from SQL Server...
  • Nicely formatted and properly mapped XML (e.g. no <row> elements as found in FOR XML RAW mode)
  • To be able to easily map columns to XML elements and attributes
  • A single root node, so I can load it into an XmlDocument without having to create the root node myself
  • Nested child elements
  • Not to have to turn my elegant little query into a huge mess of esoteric T-SQL (as with [Explicit!1!Mode])
http://richarddingwall.name/2008/08/26/nested-xml-results-from-sql-server-with-for-xml-path/

Tuesday, May 4, 2010

Configure SQL Server Agent Mail to Use Database Mail

I had an issue this morning when email notifications were not working for a SQL Server Agent Job. I found this entry in the error logs:
 
[264] An attempt was made to send an email when no email session has been established
 
A quick Google search led me to this article:
 
 
To set up SQL Server Agent Mail to use Database Mail
  1. In Object Explorer, expand a server.

  2. Right-click SQL Server Agent, and then click Properties.

  3. Click Alert System.

  4. Select Enable Mail Profile.

  5. In the Mail system list, select Database Mail.

  6. In the Mail profile list, select a mail profile for Database Mail.

  7. Restart SQL Server Agent.

Following these steps and configuring the mail profile accordingly allowed me to be able to successfully send the email notifications.

Sunday, December 28, 2008

SQL Server 2008 Management Studio

Here is the error I received:

TITLE: Microsoft SQL Server Management Studio
------------------------------

Failed to retrieve data for this request. (Microsoft.SqlServer.Management.Sdk.Sfc)

For help, click: http://go.microsoft.com/fwlink?ProdName=Microsoft+SQL+Server&LinkId=20476

------------------------------
ADDITIONAL INFORMATION:

An exception occurred while executing a Transact-SQL statement or batch. (Microsoft.SqlServer.ConnectionInfo)

------------------------------

The server principal "UserName" is not able to access the database "DatabaseName" under the current security context. (Microsoft SQL Server, Error: 916)

For help, click: http://go.microsoft.com/fwlink?ProdName=Microsoft+SQL+Server&ProdVer=09.00.3239&EvtSrc=MSSQLServer&
EvtID=916&LinkId=20476

------------------------------
BUTTONS:

OK
------------------------------
 
After doing some digging I found the answer on Aaron Bertrand's SQL blog.
 
 
"For those of you who connect to database servers where you are not in full control over all databases, or where some of your databases are ever offline (or auto-closed), the new version of Management Studio that is shipping with SQL Server 2008 is going to bring you some surprises, when you try to present a list of databases in Object Explorer."
 
"It seems that the ability to do so is hinged upon the columns that are set up in Object Explorer Details by default.  In the case I came across yesterday, the offender was "Collation."  The problem is that for a database that is offline or has been auto-closed, collation comes back as NULL.  Well, that's not really the problem... the real problem is that SSMS throws its hands in the air when it comes across NULL for these values, and assumes this is NULL because you don't have permission."
 
"And then refuses to show the data for ANY database, instead of just hiding the one(s) that caused the error.  And this is true whether or not you have Object Explorer Details even open (my guess here is that the contents of OED are cached behind the scenes, even when it is disabled... which I speculate may be part of the reason behind the sluggishness that many have complained about).  You can read more about this in Connect #354322 and in Connect #354291."
 
"For those of you that connect to databases that are hosted by 3rd party providers, some of which are accustomed to leaving as many databases in auto-close mode as possible, you are first going to have to go to Object Explorer Details, right-click the column header list, and un-check Collation.  At this point, if you refresh the Databases node in Object Explorer, you should again see all of the databases on the server, even those where you don't have access.  Strangely enough, OED still shows you some other data by default, that it probably shouldn't (e.g. recovery model, last backup, owner)."

Thursday, October 30, 2008

SQL Server 2008 Designer Behavior Change: Saving Changes Not Permitted

Saving changes is not permitted. The changes you have made require the following tables to be dropped and re-created. You have either made changes to a table that can't be re-created or enabled the option Prevent saving changes that require the table to be re-created.
 
 
"This is by design and can be quickly fixed in Management Studio by unchecking a property. To fix this in Management Studio, go to Tools -> Options then go to the Designer Page and uncheck Prevent saving changes that require table re-creation."

Tuesday, October 21, 2008

New Date Data Types in Microsoft SQL Server 2008

Here's another good article article written by Scott Mitchell discussing the new date data types available in Microsoft SQL Server 2008.
 
"This article explores the time and date data types and shows how they can be used and formatted from within an ASP.NET page. This article also includes a short discussion on the datetime2 and datetimeoffset and compares and constrasts SQL Server 2008's six different date data types."
 
 
Scott Mitchell is the the editor, founder, and primary contributor to 4GuysFromRolla.com.

Thursday, September 11, 2008

Granting execute permissions to all stored procedures in a database

"Whilst SQL Server 2000 has fixed database roles such as db_datareader and db_datawriter that allow a user read or write access respectively to all the table is a database, no such role exists for the execution of stored procedures (a db_executor role if you will). This article describes how to grant execute permission to all stored procedures in a database to a specific user or role in both SQL2000 and SQL2005."

http://www.sqldbatips.com/showarticle.asp?ID=8

Tuesday, August 12, 2008

Displaying the Sizes of Your SQL Server's Database's Tables

 
"SQL Server has a handy little system stored procedure named sp_spaceused that will return the space used by a database or by a particular table. To determine the size used by the database, simply run:"
 
EXEC sp_spaceused
 
"This will return two result sets, the first one containing the database name, size, and unallocated space and the second containing a breakdown of the database's size into how much size is reserved and how much of that is taken up by data, how much by indexes, and how much remains unused."
 
"To return information about a particular table, simply call sp_spaceused passing in as the first parameter the name of the table. To determine the space used by the Orders table in the Northwind database use:"
 
EXEC sp_spaceused 'Orders'
 
This will return a single result set that provides the following information:
  • Name - the name of the table
  • Rows - the number of rows in the table
  • Reserved - amount of total reserved space for the table
  • Data - amount of space used by the data for the table
  • Index_Size - amount of space used by the table's indexes
  • Unused - amount of usused space in the table

"While sp_spaceused can be used to return the space usage for a single table, more than likely we want a web page where we can view the space used for all tables in a database. There are a number of ways to accomplish this, but in short we need to execute sp_spaceused once for every table in the database. In order to accomplish that we could do one of two things:"

  • Query the sysobjects table to get a list of tables in the database, then use a CURSOR to iterate through these table results one at a time, executing sp_spaceused for each one.
  • Use the undocumented sp_MSforeachtable stored procedure, which takes in a command and executes that command against all of the user tables in the database.

"The sp_MSforeachtable stored procedure is one of many undocumented stored procedures tucked away in the depths of SQL Server. A list of these handy stored procedures can be found at SQL Server 2000 Useful Undocumented Stored Procedures. In short, you can use sp_MSforeachtable like so:"

EXEC sp_MSforeachtable @command1="command to run"

"In the command to run put a ? where you want the table name to be inserted. For example, to run the sp_spaceused stored procedure for each table in the database, we'd use:"

EXEC sp_MSforeachtable @command1="EXEC sp_spaceused '?'"

This will execute EXEC sp_spaceused 'TableName' for each user table in the database.
 
"An alternative option is to create a stored procedure from which a temporary table is created and populated with one record for each result set returned by sp_spaceused. The entire results, then, can be returned to the ASP.NET page as a single result set by selecting the entire contents of the temporary table at the end of the stored procedure."

create table #spaceused (name varchar(128), rows bigint, reserved varchar(25), data varchar(25), index_size varchar(25), unused varchar(25))

EXEC sp_MSforeachtable @command1="insert into #spaceused EXEC sp_spaceused '?'"

select * from #spaceused

drop table #spaceused

The above SQL code courtesy of Toby.
 
Scott Mitchell is the the editor, founder, and primary contributor to 4GuysFromRolla.com.

Thursday, July 31, 2008

Scheduling Backups for SQL Server 2005 Express

"One problem with SQL Server 2005 Express is that it does not offer a way to schedule jobs. In a previous tip, Free Job Scheduling Tool for SQL Server Express and MSDE, we looked at a free tool that allows you to create scheduled jobs for SQL Server. The one issue people often face though is what to install and what not to install on their production servers and therefore these items go without resolution. One very important part of managing SQL Server is to ensure you run backups on a set schedule. I often hear about corrupt databases and no backups, so let's take a look at another approach of scheduling backups using the included tools in both the operating system and SQL Server."
 
This article provides an easy solution for using the SQL BACKUP command to backup a database and shows you how to use SQLCMD.EXE with Scheduled Tasks to schedule it to run automatically, thus giving you the scheduled backups functionality you would typically have in SQL Server Agent.
 

Thursday, May 1, 2008

Temporary Tables and Dynamic SQL

When programming in SQL sometimes you need to generate temporary tables inside a stored procedure. That's pretty straight forward with code like this:
 
SELECT     TrackingGroupID,
               
Tag
INTO        #temp
FROM       MyTable
 
And you can access #temp in the rest of your stored procedure and drop the temp table when complete.
 
-- testing
SELECT COUNT(*) FROM #temp
 
-- drop temp table
DROP TABLE #temp
 
But if you ever have a situation where you also need to use dynamic SQL you will likely have scope issues. In my particular case I ran into a situation where I needed to apply a dynamic filter to a query and store that data in a temporary table so I could access it after the filter was applied and do further manipulation and analysis and grouping with the data.
 
Note: Typically when I work with temp tables I try to use tables-variables like so:
 
-- create temp table
DECLARE @Temp TABLE
(
    
TrackingGroupID  INT,
    
Tag                    VARCHAR(50) NOT NULL DEFAULT ''
)
 
However, in this case I was not able to get a table-variable to work. So I ended up using a #temp table instead.
 
My first attempt looked something like this:
 
-- declare variables
DECLARE @SQL VARCHAR(8000)
 
-- apply filter
SET @SQL = 'SELECT      TrackingGroupID,
                                  
Tag
                   
INTO         #temp
                  
FROM        MyTable
                  
WHERE       1=1 '
+ @Where + '
                  
GROUP BY  TrackingGroupID, Tag'

-- apply filter
EXEC(@SQL)

-- do further manipulation or analysis on #temp table
....


-- drop temp table
DROP TABLE #temp
 
But I received this error:
 
Msg 208, Level 16, State 0, Line 18
Invalid object name '#temp'.
 
I knew it was a scoping issue, but I wasn't sure how to work around it exactly. Researching on Google I came across this article:
 
 
"The problem here is the scope of the session. When we execute dynamic sql via EXEC or sp_executesql a new scope is created for a child session. Any objects created in that session are dropped as soon as the session is closed."
 
So I modified my code like so:
 
-- create temp table
CREATE TABLE #temp
(
    
TrackingGroupID  INT,
    
Tag                    VARCHAR(50) NOT NULL DEFAULT ''
)
 
-- declare variables
DECLARE @SQL VARCHAR(8000)
 
-- apply filter
SET @SQL = 'INSERT INTO #temp
                       
(TrackingGroupID, Tag)
                       
SELECT       TrackingGroupID,
                                        
Tag
                       
FROM         MyTable
                       
WHERE       1=1 '
+ @Where + '
                       
GROUP BY  TrackingGroupID, Tag'
 
-- apply filter
EXEC(@SQL)
 
-- do further manipulation or analysis on #temp table
....
 
-- drop temp table
DROP TABLE #temp
 
Now I had access to the #temp table throughout the scope of the stored procedure!
 
I've been told that ##temp might have solved it also since it's global; however, I'm not as familiar with ## so I didn't go down that road.