Thursday, September 04, 2008

MS Excel calling into Haskell


Recently, I was tasked with creating an Excel addin that calls into a Haskell module. While most of the pieces are already well-documented, I haven't seen them put all together. Furthermore, there additional modifications that are needed to get this to work. For this end, I decided to put up a quick how-to on this.


The Haskell environment


I used ghc 6.8.3 for this project. In the docs, they have some pretty good notes about making a DLL. I say pretty good, because even with minor version changes (6.6.1 to 6.8.3), this seems to have changed a bit. After finding the right version of the docs, things went much more smoothly. Once found, I pretty much followed the steps verbatim from :

12.6.2. Making DLLs to be called from other languages


The Haskell DLL


Verbatim from the docs, the actual Haskell source file, adder.hs:


module Adder where

adder :: Int -> Int -> IO Int -- gratuitous use of IO
adder x y = return (x+y)

foreign export stdcall adder :: Int -> Int -> IO Int



And the corresponding DllMain.c that will be linked together with:


#include <windows.h>
#include <rts.h>

extern void __stginit_Adder(void);

static char* args[] = { "ghcDll", NULL };
/* N.B. argv arrays must end with NULL */

BOOL STDCALL
DllMain(
HANDLE hModule,
DWORD reason,
void* reserved)
{
return TRUE;
}

__stdcall void AdderBegin()
{
startupHaskell(1, args, __stginit_Adder);
}

__stdcall void AdderEnd()
{
hs_exit();
}



Note, that this code is similiar but different then the one at the sample page

noted above. Mainly, because of the DllMain issues, we don't call anything in the DllMain.


The export header file:



#ifdef __cplusplus
extern "C"
{
#endif
__declspec(dllexport) void __stdcall AdderBegin(void);
__declspec(dllexport) void __stdcall AdderEnd(void);
__declspec(dllexport) long __stdcall adder(long x, long y);
#ifdef __cplusplus
}
#endif



In some cases (noted below), a DEF file is needed, like so:


LIBRARY Adder
EXPORTS
adder@8=_adder
AdderBegin@0=_AdderBegin
AdderEnd@0=_AdderEnd



To help with debugging, a Test console application:



#ifdef __cplusplus
#include "stdafx.h"
#include "TestConsole.h"
#include "../Adder/adder.h"

#ifdef _DEBUG
#define new DEBUG_NEW
#endif

// The one and only application object

CWinApp theApp;

using namespace std;

int _tmain(int argc, TCHAR* argv[], TCHAR* envp[])
{
int nRetCode = 0;

// initialize MFC and print and error on failure
if (!AfxWinInit(::GetModuleHandle(NULL), NULL, ::GetCommandLine(), 0))
{
_tprintf(_T("Fatal Error: MFC initialization failed\n"));
nRetCode = 1;
}
else
{
AdderBegin();

wcout << adder(5, 3);
wcout << _T("\r\n");

AdderEnd();
}

return nRetCode;
}

Finally, the MS Excel addin:

The excel addin is based off of the article at:
http://blogs.msdn.com/andreww/archive/2007/12/09/building-an-excel-xll-in-c-c-with-vs-2008.aspx
Although this project is created to be a Excel 2007 addin, there is no inherit functionality related to 2007 and with some minor modifications should work off older versions of excel.

I have changed



__declspec(dllexport) long SumTwo(long arg1, long arg2)
{
return adder(arg1, arg2);
}

// Excel calls xlAutoOpen when it loads the XLL.
__declspec(dllexport) int WINAPI xlAutoOpen(void)
{
static XLOPER12 xDLL;
int i;

AdderBegin();

// Fetch the name of this XLL. This is used as the first arg
// to the REGISTER function to specify the name of the XLL.
Excel12f(xlGetName, &xDLL, 0);

// Loop through the g_rgUDFs[] table, registering each
// function in the table using xlfRegister.
for (i = 0; i < g_UdfCount; i++)
{
Excel12f(xlfRegister, 0, 1 + g_UdfDataFieldCount,
(LPXLOPER12) &xDLL,
(LPXLOPER12) TempStr12(g_rgUDFs[i][0]),
(LPXLOPER12) TempStr12(g_rgUDFs[i][1]),
(LPXLOPER12) TempStr12(g_rgUDFs[i][2]),
(LPXLOPER12) TempStr12(g_rgUDFs[i][3]),
(LPXLOPER12) TempStr12(g_rgUDFs[i][4]),
(LPXLOPER12) TempStr12(g_rgUDFs[i][5]),
(LPXLOPER12) TempStr12(g_rgUDFs[i][6]),
(LPXLOPER12) TempStr12(g_rgUDFs[i][7]),
(LPXLOPER12) TempStr12(g_rgUDFs[i][8]),
(LPXLOPER12) TempStr12(g_rgUDFs[i][9]),
(LPXLOPER12) TempStr12(g_rgUDFs[i][10])
);
}

// Free the XLL filename.
Excel12f(xlFree, 0, 1, (LPXLOPER12) &xDLL);
return 1;
}

// Excel calls xlAutoClose when it unloads the XLL.
__declspec(dllexport) int WINAPI xlAutoClose(void)
{
int i;
//debugPrintf("xlAutoClose\n");

// Delete all names added by xlAutoOpen or xlAutoRegister.
for (i = 0; i < g_UdfCount; i++)
{
Excel12f(xlfSetName, 0, 1, TempStr12(g_rgUDFs[i][2]));
}

AdderEnd();
return 1;
}



A build script to put it all together:


cd adder
ghc -c adder.hs -fglasgow-exts
ghc -c dllMain.c
ghc -shared -o adder.dll adder.o adder_stub.o dllMain.o
rem ghc -shared --enable-stdcall-fixup -o adder.dll adder.o adder_stub.o dllMain.o
lib /MACHINE:x86 /DEF:adder.def /OUT:adder.lib /NOLOGO /SUBSYSTEM:WINDOWS
copy adder.lib ..\Bin\Lib
copy adder.dll \windows
cd ..
msbuild ExcelToHaskell.sln /t:Rebuild /p:Configuration=Release /p:Platform=Win32

 


Dumpbin to the rescue


Testing the Haskell module with the TestConsole app is quite helpful. Initially, I was having some linking problems. Using the SDK tool dumpbin /exports was quite helpful in debugging.


VS 2005 / 2008 differences


Interesting enough, some minor changes are needed if you are upgrading to VS 2008 or switching between this and VS 2005. The 2008 version seems to need the DEF file while the 2005 one doesn't.


Putting the DLL in the right place


Normally, putting DLLs in the same directory as the module that are calling them is enough for them to be found. In this case, however, the calling process is Excel and hence the DLL needs to be somewhere on the standard calling path. In this case, I just made a quick and dirty solution of putting the module in the WINDOWS directory.

Monday, July 28, 2008

MsBuild and Installer Projects

I remember the first time I tried msbuild. I read about it and then found out that it can actually work with the VS generated solution files. Great. Seemed easy enough. But when I tried it, I got something like the following:

Project "C:\ZenUtils.sln" on node 0 (default targets).
Building solution configuration "DebugDefault".C:\ZenUtils.sln : warning MSB4078: The project file "ZenUtilsSetup\ZenUtilsSetup.vdproj" is not supported by MSBuild and cannot be built.


Hmmmmn. Not good. The short of it is the developers of msbuild never took into account Installer projects (.vdproj files). While this may change in the future, for now this is what we have. Unfortunately, it also takes msbuild out of the picture as the one-step master builder. You will need to run a solutions containing installer projects separately with devenv.com. Yes, this is a cumbersome step.

To minimize using devenv and maximize using msbuild, I do the following:

  • Create 2 solutions. One is for my compiled components and the other is for my installer project(s)
  • In the installer project, I manually add the compiled components back in, instead of using the project outputs. I lose some project references in the installer project, but as I found quarky in the first place, it wasn't really a loss.
  • I also manually add back in needed merge modules. In my case, Microsoft_VC80_CRT_x86.msm for the common c/c++ runtime and Microsoft_VC80_MFC_x86.msm for MFC support and their associated policy files.
  • Then I include the following tasks in my nant build file:

<target description="msbuild" name="MsBuildCompile">

    <echo message="Building Binaries">

   <msbuild project="..\..\Src\ZenUtils.sln">

        <arg value="/property:Configuration=Release">

        <arg value="/t:Rebuild">

</target>

<target name="BuildMsi">

    <exec

        program="devenv.com"

        basedir="C:/Program Files/Microsoft Visual Studio 8\Common7/IDE"
        workingdir="..\..\Src\Product"

       commandline="ZenUtilsInstaller.sln

         /Rebuild Release">

    </exec>

</target>

Automating the Build Process

This is the first in a series of articles about automating the build process. While the specifics often involve Windows and VS, hopefully this information is generic to applied cross-platform. The basics have been covered quite well elsewhere. One good resource is http://www.codeplex.com/treesurgeon/Wiki/View.aspx?title=DevelopmentTreeIntroduction&referring.

Briefly, essential tools:

  • Compiler, Editor - VS - Actually not that essential, but is in quite common use and includes all the tools needed for compiling and linking. Even those people who use alternative tools, usually know something about how VS works.
  • Source control - SVN - quite popular for the last few years and there is a reason for it. Easy to use and integrates well.
  • Builder - msbuild - will control the building of the components, include with .NET framework.
  • Builder - nant - will control the building of the components

It seems like there is sometimes a debate of which one to use, nant or msbuild, with various camps lining up on one side or the other. My solution is to use them both. More details, later, but in short, I like using msbuild and just give it my automatically made VS solution file and away it goes. I wrap this task (and others) in a nant build file, which is a little bit friendlier to hand edit.

  • Unit Testing - nunit - For .NET components
  • Unit Testing - UnitTest++ - Currently the one I like for c++

Directory structure

Having a consistent directory structure across components and projects is essential. I can't say more then already mentioned article and it's associated project, Tree surgeon http://www.codeplex.com/treesurgeon.

Saturday, March 04, 2006

Using Ant's dotnet lib

The Ant framework now has support for using dotnet tools. Of particular interest, is support for msbuild, which is actually a similiar tool as Ant with Microsoft flavor to it. This can provide a lot variations for automated builds. However, there are few tricks in getting Ant's framework support for dotnet going. If you are having problems using or installing Ant's new support for dotnet, read on...

Ant's framework doesn't inherently support the ant-dotnet.jar until version 1.7.*. So, until then, you will need to patch ant to include. If you are an experienced java developer, this will be an old hat to you. If you are not that familiar with developing or compiling java components, follow the following steps (Note this assumes that have the java compiler installed and in your path):

1. grab this (http://cvs.apache.org/~bodewig/dotnet/ant-dotnet.jar) and place in the same dir as your other ant jars (assuming that you have set an environment variable ANT_HOME, when you installed ant, this would be %ANT_HOME%\lib) Note: My system kept on downloading the file as a '.zip'. It is actually a .jar. Rename as such if necessary.

2. . Also grab (this) XMLFragment.java java file. This is file that you will need to compile and add it into the main ant.jar You can do this by the following:

3. setup a src directory, to the effect of

<basedir>\apache\tools\ant\util

Basedir can be any dir, but the directory structure beneath must be the same.

4. copy the xmlfragment.java file into \org\apache\ant\dotnet\tools\ant\util

5. at a command prompt:

cd <basedir>basedir>\org\apache\ant\dotnet\tools\ant\util

6. javac -classpath %ANT_HOME%\lib\ant.jar XMLFragment.java

7. cd <basedir>

8. jar uvf ant.jar org\apache\tools\ant\XMLFragment.class

9. copy /y ant.jar c:\Java\apache-ant-1.6.5\lib

That's all there is to it.