Blog

Useful tidbits related to software development, that I think might be of use or interest to everyone else (or to me when I forget what I did!)

ASP.NET URL Rewriting and Tilde

November 04, 2008

In ASP.NET controls you can use the tilde '~' character to signify the virtual application root of the website, for example when sourcing an image for an ImageButton. Code:
  <asp:ImageButton ID="ImageButton1" runat="server" AlternateText="Example Button" ImageUrl="~/images/button.png" />
This is fine for most cases. However, when using URL rewriting (via HttpHandlers) it can become a problem, since ASP.NET will calculate the path to the resource based on the location of the physical page it is rendering (not the page requesed in the browser [RawUrl]). If the resource is in a directory below the page directory, then it will render a relative path to the HTML, (and not an absolute path from the app root). When your browser sees a relative path in the HTML it appends that to the current working directory to retreive the resource, which for a page being served up via a rewrite, is probably the wrong directory. For example, imagine you have the following directory structure: /MyButtonPage.aspx /images/button.png /subdir/page-served-by-httphandler-mapping-to-mybuttonpage.aspx If you view '/MyButtonPage.aspx' (containing the code from above) the output path to the image would be 'images/button.png'. Your browser will therefore retreive '/images/button.png' which is correct. However, if you view '/subdir/page-served-by-httphandler-mapping-to-mybuttonpage.aspx' (which rewrites internally the '/MyButtonPage.aspx') then ASP.NET will again render the image path as 'images/button.png' (since it is relative to the 'MyButtonPage.aspx' which was rendered internally). Your browser will now try to retrieve, '/subdir/images/button.png' which does not exist. A workaround for this problem is to not use the tilde mechanism and instead reference the absolute path to the resource in the code. e.g.
  <asp:ImageButton ID="ImageButton1" runat="server" AlternateText="Example Button" ImageUrl="/images/button.png" />
However, this will reference from the website root, not the application root, so be careful if your site runs as a virtual application within another site.

Safe Rewriting URL Encoder

October 30, 2008

When using URL rewriting to make nice URLs without querystrings, it is useful to include a meaningful name for the page, i.e. the page title, in the URL itself. This helps usability in distinguishing pages and also aids search engines in spidering different content that may be generated by a single page on your server. For some reason, the HttpUtility.UrlEncode and HttpUtility.HtmlEncode just dont cut it for making good URLs that are readable, optimized and work on the browser/server. I have written my own class for cleaning/encoding text into a form that can be used as part of a URL that will be used in a rewriting engine. Code:
Imports System.Text.RegularExpressions

Public Class UrlEncoder
    Public Shared Function EncodeRewriteUrl(ByVal inputString As String) As String
        Dim url As String = Regex.Replace(System.Web.HttpUtility.HtmlDecode(inputString.Replace(" ", "-")), "[^a-zA-Z0-9\-]", "")
        While url.Contains("--")
            url = url.Replace("--", "-")
        End While
        Return url
    End Function
End Class
The idea here is that you pass in the text you want to use as your URL, and you receive a hyphenated URL containing only URL safe characters ready for use with your rewriting engine. Example: if.. pageId=1 pageTitle="A page about 'dogs' and 'cats' (also other things)" and.. pageUrl=UrlEncoder.EncodeRewriteUrl(pageTitle & "-" & pageId & ".aspx") then.. The page URL would be: A-page-about-dogs-and-cats-also-other-things-1.aspx

Blogger extensions and integration using ASP.NET

October 27, 2008

After recently starting this blog using my Blogger account and its FTP publishing feature I wanted a more seamless integration with my existing site for the index page of the blog. In other words, I wanted to show the latest blogs as part of my existing site using the masterpage I have in ASP.NET. To achive this goal, I have set Blogger to publish to /blog on my domain, with a proprietary filename and .html extension. I have then put a Default.aspx page in the /blog directory, which uses the site master page. This default page basically opens the .html file server side and writes the stripped out relavent HTML content to a placeholder in the ASP.NET page. I have included some of the Blogger specific meta/link tags in the .aspx page head, to enable the RSS/Atom feeds from the page and to enable the edit post controls etc. to work. As its ASP.NET you can also add extra features to the page. I have added a drop down list of the categories (Blogger labels) to use as a quick link to each category. Blogger.com creates a .html page for each of your labels containing links to the relevant posts, these files are stored under the 'labels' folder, which can be simply enumerated on the server. Code:
if not IsPostback then

   try

    Dim file As String = "blog-content.html"

    Response.Cache.SetLastModified(IO.File.GetLastWriteTime(Server.MapPath(file)))

    dim content as string=IO.File.ReadAllText(Server.MapPath(file))
    content = content.Substring(content.IndexOf("<body>") + "<body>".Length, content.LastIndexOf("</body>") - content.IndexOf("<body>") - "<body>".Length)
    content = content.Remove(content.IndexOf("<iframe"), content.IndexOf("</iframe>") + "</iframe>".Length - content.IndexOf("<iframe"))

    lblContent.Text=content

    'load the labels
    dim labelFiles() as string=IO.Directory.GetFiles(Server.MapPath("labels"))

    ddlCats.Items.Clear
    ddlCats.Items.Add(New ListItem("--Jump To Category--",""))  

    for each filename as string in labelFiles

     dim url as string=filename.substring(filename.lastindexof("\")+1,filename.length-filename.lastindexof("\")-1)

     ddlCats.Items.Add(New ListItem(url.replace(".html",""),url))

    next

   catch ex as exception

    lblContent.Text="Error: " & ex.Message

   end try
My site only uses this method for the index page, but you could extend this idea even further to display all blog pages within your custom ASP.NET / masterpage site, by putting the /blog directory under a custom HttpHandler (if your server allows this) and rewriting the URL to your ASP.NET page. Also I would suggest using regular expressions for filtering the HTML if your server supports it. -- Edit: 25/11/2008 I have now improved the integration by using the HTML generated by my masterpage as the template on Blogger.com, so all the pages look like you are on my website. This means I could refine the above process of finding the correct content for each placeholder, (with the addition of a seperate placeholder for the sidebar) as the divs that contain the data have specific class names. I have also created two more aspx pages, "index by date" and "index by category" - the code behind for these simply enumerate the "archives"/"labels" folders on the server, (the same as my drop down box above). Additionally, these pages open each .html file to parse the content for all of the post titles and permalinks to create the list of links for each date/label. -- Edit: 18/10/2010 Since my blog no longer uses Blogger due to FTP publishing being cancelled this code is now redundant.

Programmatically Setting the Default Button of a Textbox

October 09, 2008

In ASP.NET, setting the default button of a textbox is usually a simple task as you simply wrap the button and textbox in an asp:panel and set the 'defaultbutton' attribute on the panel. Sometimes however, it is not possible to put the button and textbox in a panel together, for example if the textbox is part of a databound template (e.g. gridview, repeater etc.) If you look at the HTML code that the asp:panel method generates, it basically attaches a javascript event handler to the 'onkeypress' of the textbox. I have wrapped this functionality, so that you can programmatically set the default button of any textbox control on your page, to any button on your page. Code:
Public Shared Sub SetDefaultButton(ByVal textbox As System.Web.UI.WebControls.TextBox, ByVal button As System.Web.UI.WebControls.IButtonControl)
 If TypeOf button Is System.Web.UI.WebControls.WebControl Then
 textbox.Attributes("onkeypress") = "javascript:if (event.keyCode == 13){ document.getElementById('" & CType(button, System.Web.UI.WebControls.WebControl).ClientID & "').click();event.cancelBubble = true;if (event.stopPropagation) event.stopPropagation();return false;}"
 End If

End Sub

Using .NET TripleDES Crypto Provider

August 28, 2008

I recently wrote a wrapper class implementing the .NET System.Security.Cryptography.TripleDESCryptoServiceProvider to quickly allow you to encrypt a plaintext string and return the Base64 encoded string of the encrypted bytes created by the TripleDES cipher (and decrypt that string back to plaintext!). This provides a very simple way to encrypt and decrypt your data without the need to store it as a binary image. The VB.NET code is shown below:
Public Class TripleDES
        Private _k, _iv As Byte()

        Public Sub New(ByVal key As Byte(), ByVal iv As Byte())
            _k = key
            _iv = iv
        End Sub

        Public Property Key() As Byte()
            Get
                Return _k
            End Get
            Set(ByVal value As Byte())
                _k = value
            End Set
        End Property

        Public Property IV() As Byte()
            Get
                Return _iv
            End Get
            Set(ByVal value As Byte())
                _iv = value
            End Set
        End Property

        Public Function Encrypt(ByVal plaintext As String) As String
            Dim cipher As New System.Security.Cryptography.TripleDESCryptoServiceProvider()
            cipher.BlockSize = 64

            Dim cipherText As String
            Using cipherTextStream As New IO.MemoryStream()
                Using cryptoStream As New System.Security.Cryptography.CryptoStream(cipherTextStream, cipher.CreateEncryptor(_k, _iv), System.Security.Cryptography.CryptoStreamMode.Write)
                    Dim bytes As Byte() = Text.Encoding.UTF8.GetBytes(plaintext)
                    cryptoStream.Write(bytes, 0, bytes.Length)
                    cryptoStream.FlushFinalBlock()

                    cipherText = Convert.ToBase64String(cipherTextStream.ToArray())
                End Using
            End Using

            Return cipherText

        End Function

        Public Function Decrypt(ByVal ciphertext As String) As String
            Dim cipher As New System.Security.Cryptography.TripleDESCryptoServiceProvider()
            cipher.BlockSize = 64

            Dim plaintext As String
            Using plaintextStream As New IO.MemoryStream()
                Using cryptoStream As New System.Security.Cryptography.CryptoStream(plaintextStream, cipher.CreateDecryptor(_k, _iv), System.Security.Cryptography.CryptoStreamMode.Write)
                    Dim bytes As Byte() = Convert.FromBase64String(ciphertext)
                    cryptoStream.Write(bytes, 0, bytes.Length)
                    cryptoStream.FlushFinalBlock()

                    plaintext = Text.Encoding.UTF8.GetString(plaintextStream.ToArray())
                End Using
            End Using

            Return plaintext

        End Function
    End Class
To use it, simply instanciate it with your Key and Initialization Vector. Make sure your K and IV are the same when decrypting data you encrypted with them!

Simple Password Generator

August 28, 2008

To generate random passwords I have written a simple class in VB.NET, which accepts a string containing valid password characters and then generates passwords of n length using the specified characters.
Public Class PasswordGenerator
        Private _pwChars As String

        Public Sub New()
            MyClass.New("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789")
        End Sub

        Public Sub New(ByVal validChars As String)
            _pwChars = validChars
        End Sub

        Public Function GeneratePassword(ByVal length As Integer) As String
            Dim password As String = ""
            Dim rndNum As New Random()

            For i As Integer = 1 To length
                password &amp;= _pwChars.Substring(rndNum.Next(0, _pwChars.Length - 1), 1)
            Next

            Return password
        End Function

    End Class

Thread.Join Deadlock with Invoke

September 07, 2007

If you are trying to Make Thread-Safe Calls to Windows Forms Controls then you can run into deadlock problems when calling 'Invoke' from your thread, if your main thread is waiting for a Thread.Join on the calling thread. I posted a workaround for this on the Microsoft site, which uses an extra thread to make the call to Invoke method, to avoid the deadlock. VB.NET example:
Private Sub SetText(ByVal str_text As String)
        If Me.textbox1.InvokeRequired = True Then
                'call an asyncronous invoke, by calling it then forcing a DoEvents
                Dim asyncInvokeThread As New Threading.Thread(New Threading.ParameterizedThreadStart(AddressOf AsyncInvoke))
                asyncInvokeThread.IsBackground = True
                asyncInvokeThread.Start(str_text)

                Application.DoEvents()
        Else
                me.textbox1.text=str_text
        End If
End Sub
Private Sub AsyncInvoke(ByVal obj_text As Object)
        Dim str_text As String = CStr(obj_text)
        Dim d As New SetTextCallback(AddressOf SetText)
        Me.Invoke(d, New Object() {str_message})
End Sub

x86 Assembly Base64 Encoder

September 19, 2006

I joined in a "competition" with some of the experts in the assembly forums over at experts-exchange to write the fastest assembly code Base64 encoder, following the guidelines of the RFC3548. Several versions emerged, some using lookup tables to precompute results, which faired well on CPUs which large caches. My version was more about opcode optimization and using bitwise arithmetic over decimal arithmetic, which achieved best results on CPUs that have better pipelining. Code:
  void ToBase64( BYTE* pSrc, char* pszOutBuf, int len )
{
      char* chr_table="ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";

       __asm{
                  mov ecx, len
                  mov esi, pSrc                              //;bytes from source
                  mov edi, chr_table
                  push ebp
                  mov ebp, pszOutBuf

src_byteLoop:

                  xor eax, eax

                  //;read 3 bytes
                  mov ah, byte ptr[esi]
                  mov al, byte ptr[esi+1]
                  shl eax, 16
                  mov ah, byte ptr[esi+2]

                  //;manipulate in edx bitset1
                  mov edx, eax
                  shl eax, 6                                    //;done first 6 bits

                  shr edx, 26            
                  mov bl, byte ptr [edi+edx]            //;put char in buffer
                  mov byte ptr[ebp], bl
                  inc ebp                                          //;next buf

                  //;manipulate in edx bitset2
                  mov edx, eax
                  shl eax, 6                                    //;done first 6 bits

                  shr edx, 26
                  mov bl, byte ptr [edi+edx]            //;put char in buffer
                  mov byte ptr[ebp], bl
                  inc ebp                                          //;next buf

                  //;manipulate in edx bitset3
                  mov edx, eax
                  shl eax, 6                                    //;done first 6 bits

                  shr edx, 26
                  mov bl, byte ptr [edi+edx]            //;put char in buffer
                  mov byte ptr[ebp], bl
                  inc ebp                                          //;next buf

                  //;manipulate in edx bitset4
                  mov edx, eax
                  shl eax, 6                                    //;done first 6 bits

                  shr edx, 26
                  mov bl, byte ptr [edi+edx]            //;put char in buffer
                  mov byte ptr[ebp], bl
                  inc ebp                                          //;next buf

                  //;done these bytes
                  add esi, 3
                  sub ecx, 3

                  cmp ecx, 3
                  jge src_byteLoop                        //;still got src bytes

                  xor eax, eax                              //;set to zero (pad count)
                  cmp ecx, 0
                  jz finished

                        //;need to pad out some extra bytes

                        //;read in 3 bytes regardless of junk data following pSrc - already zero from above)
                        mov ah, byte ptr[esi]
                        mov al, byte ptr[esi+1]
                        shl eax, 16
                        mov ah, byte ptr[esi+2]

                        sub ecx, 3                                    //;bytes just read
                        neg ecx                                          //;+ve inverse
                        mov edx, ecx                              //;save how many bytes need padding

                        //;as per the RFC, any padded bytes should be 0s
                        mov esi, 0xFFFFFF
                        lea ecx, dword ptr[ecx*8+8]            //;calculate bitmask to shift
                        shl esi, cl
                        and eax, esi                              //;mask out the junk bytes

                        mov ecx, edx                              //;restore pad count

                        //;manipulate in edx byte 1
                        mov edx, eax
                        shl eax, 6                                    //;done first 6 bits                        

                        shr edx, 26
                        mov bl, byte ptr [edi+edx]            //;put char in buffer
                        mov byte ptr[ebp], bl
                        inc ebp                                          //;next buf

                        //;manipulate in edx byte 2
                        mov edx, eax
                        shl eax, 6                                    //;done first 6 bits                        

                        shr edx, 26
                        mov bl, byte ptr [edi+edx]            //;put char in buffer
                        mov byte ptr[ebp], bl
                        inc ebp                                          //;next buf

                        //;manipulate in edx byte 3
                        mov edx, eax
                        shl eax, 6                                    //;done first 6 bits                        

                        shr edx, 26
                        mov bl, byte ptr [edi+edx]            //;put char in buffer
                        mov byte ptr[ebp], bl
                        inc ebp                                          //;next buf

                        //;manipulate in edx byte 3
                        mov edx, eax
                        shl eax, 6                                    //;done first 6 bits                        

                        shr edx, 26
                        mov bl, byte ptr [edi+edx]                  //;put char in buffer
                        mov byte ptr[ebp], bl
                        inc ebp                                          //;next buf

                        mov eax, ecx                              //;'return' pad count

finished:
                  test eax, eax
                  jz end
                  //;some bytes were padding, put them as =

                        sub ebp, eax                        //;move ptr back for num bytes to pad
padChars:
                        mov byte ptr[ebp], 0x3d            //;=
                        inc ebp
                        dec eax

                        jnz padChars

end:
                  pop ebp
        }      
}
There were several key points to my optimization technique:
  • Firstly unrolling of loops, which basically means eliminating any kind of loop structures in the code, in favour of manually typing the code to perform the operations on the data. In terms of asm optimizations this reduces branching and shifting EIP too often.
  • Another key point, was minimizing the load/store operations performed. To keep these down, you can use indirect addressing to get to the byte in memory you are interested in.e.g. mov edx, base add edx, edi mov bl, byte ptr[edx] becomes mov bl, [edx+edi]
  • As Base64 encoding requires you to work at bit level using 3 bytes at a time, you can speed up the conversion of the bit sets by using the binary arithmetic operations of SHL and SHR (shift left/right) to progressively take the most significant bits of one DWORD and operate on them with a lower significance by shifting them.For example, if EAX contains a DWORD and you copy that the EDX. You can get rid of the left most 6 bits of EAX by shifting left 6. You can deal with those 6 bits in EDX, by shifting EDX right 26 times. This is much quicker than shifting out one bit at a time.
My final throughput on the official test machine was 196.92 MB/s, which is almost double the throughput of the CryptoAPI function, which came in at 104.05 MB/s

C++ Port Scanner using Winsock with Timeout

July 30, 2005

When writing the UltraChat program, I needed to write a port scanner that would look for running UltraChat servers. A port scanner basically attempts to connect to a host on a given port and return true or false if a connection was (or wasn't) established. The problem with doing this is that when a connection cannot be establish, Winsock (windows sockets) will keep trying, allowing for servers that are slow to reply. Which then slows the whole process down. To get around this I used a simple threading and timer technique to kill the request if it doesnt succeed within a given time. Code:
bool quickConnect(char* remoteHostIP, int port)
{
      struct sockaddr_in sAddress;

      //set up connection info
      sAddress.sin_family = AF_INET;
      sAddress.sin_port = htons(port);                                                      //port
      sAddress.sin_addr.S_un.S_addr = inet_addr(remoteHostIP);                  //IP

      //use a thread to do the connection
      HANDLE tConnThread;
      DWORD threadID;
      DWORD tExitCode=0;

      tConnThread=CreateThread(NULL,0,&quickConnectTTL,&sAddress,0,&threadID);

      //now wait 1seconds maximum for response
      SYSTEMTIME now;
      GetSystemTime(&now);

      int finishTime=(now.wDay * 24 * 60)+(now.wHour * 60)+(now.wSecond)+1;
      int nowTime=0;

      while(nowTime<finishTime){
            GetSystemTime(&now);
            nowTime=(now.wDay * 24 * 60)+(now.wHour * 60)+(now.wSecond);
            //check if already exited, dont waste time
            GetExitCodeThread(tConnThread,&tExitCode);
            if(tExitCode!=STILL_ACTIVE){
                  break;
            }
      }

      //get the return value from connection
      GetExitCodeThread(tConnThread,&tExitCode);

      //if thread did not exit, time is up, close the thread and assume no server
      if(tExitCode==STILL_ACTIVE){
            TerminateThread(tConnThread,0);
            tExitCode=0;
      }

      CloseHandle(tConnThread);

      bool present;
      //check return values
      if(tExitCode==1){
            present=true;
      }
      else{
            present=false;
      }

      return present;

}

DWORD WINAPI quickConnectTTL(LPVOID sAddressPTR){
      //copy struct info from PTR
      struct sockaddr_in sAddress;
      memcpy(&sAddress,sAddressPTR,sizeof(struct sockaddr_in));

      //connect to remote host
      int concode;
      SOCKET s=socket(AF_INET,SOCK_STREAM,0);
      concode=connect(s ,(struct sockaddr *)&sAddress,sizeof(sAddress));

      if(concode==SOCKET_ERROR){
            ExitThread(0);
            return 0;
      }
      else{
            closesocket(s);
            ExitThread(1);
            return 1;
      }
}
The above example waits 1 second for a reply, and then assumes a false result. To use the code, you need to include the windows sockets headers and initialise the winsock, then call quickConnect passing the host/port.

Extending Functionality of a compiled PE using DLLs

June 29, 2005

Sometimes it is necessary to extend the functionality of a program to which you don't have the source code. This tutorial focuses on adding functionality to a compiled Windows Portable Executable file, but the idea can probably be implemented on other compiled binaries. Adding simple functionality can be achieved directly in 'code caves' within the PE file by adding the assembly code directly to the exe to perform the operation (i.e. use a hex editor with assembler to render the opcodes directly into some empty space in the exe file) The more complex the functionality, the more assembly code this will require, which needs bigger code caves and takes more coding on your part. You can create bigger code caves by adding new sections to the file, but that is not the topic of this tutorial. An easier way, I have found, is to code the extra functionality in a DLL, which is best coded in C++/C as these are easier to import into the target app. You should use the extern "C" macro on your exports to make them asm friendly (no mangling) and specify 'declspec(dllexport)' to make the compiler put the function in the exports.. For example: (AddOns.h)
#ifdef ADDONS_EXPORTS
#define ADDONS_API extern "C" __declspec(dllexport)
#else
#define ADDONS_API __declspec(dllimport)
#endif

ADDONS_API bool someFunction(LPCTSTR someTextParam);
You can code the core of the functionality in the methods defined in your DLL and then the only assembly you need to code is loading your DLL and calling the functions. The above function for example, will be in a DLL called 'AddOns.dll' and take 1 paramater (a pointer to a string). In assembly code you can load the dll using the LoadLibrary function and find the address of the function using GetProcAddress. If the exe does not import these functions, read my other blog post on Finding the address of GetProcAddress.
push &"AddOns.dll"                ; address of string for the DLL, in a code cave
call &LoadLibraryA                ; address of the imported/loaded LoadLibraryA function
push &"someFunction"              ; address of string for the function, in a code cave
push eax                          ; HMODULE AddOns.dll (returned from LoadLibrary)
call &GetProcAddressA             ; address of the imported/loaded GetProcAddressA function
push &"SomeString"                ; address in the app to the parameter you want to pass
call eax                          ; call the function in the dll (returned from GetProcAddress)
add esp, 4                        ; clear the stack
jmp &returnAddress                    ; go back to the original code?
Since C functions use the cdecl calling convention by default, you have to clean the stack of your params when you are finished and if your function returns a value, it will be in EAX. Once you have put the above code in a cave somewhere, you simply need to jmp to it at the appropriate point in the original exe and then have it jmp back to the best place in the code to give control back to the normal program flow when you have finished.