2017-05-11
Tracing IDispatch::Invoke calls in COM applications
At Collabora Productivity we recently encountered the need to investigate calls in a third-party application to COM services offered by one or more other applications. In particular, calls through the IDispatch mechanism.
In practice, it is use of the services that Microsoft Office offers to third-party applications that we want to trace and dump symbolically.
In practice, it is use of the services that Microsoft Office offers to third-party applications that we want to trace and dump symbolically.
We looked around for existing tools but did not find anything immediately suitable, especially not anything available under an Open Source license. So we decided to hack a bit on one of the closest matches we found, which is Deviare-InProc. It is on GitHub, https://github.com/nektra/Deviare-InProc.
Deviare-InProc already includes code for much of the hardest things needed, like injecting a DLL into a target process, and hooking function calls. What we needed to do was to hook COM object creation calls and have the hook functions notice when objects that implement IDispatch are created, and then hook their Invoke implementations.
The DLL injection functionality is actually "just" part of the sample code included with Deviare-InProc. The COM tracing functionality that we wrote is based on the sample DLL to be injected.
One problem we encountered was that in some cases, we would need to trace IDispatch::Invoke calls that are made in a process that has already been started (through some unclear mechanism out of our control). The InjectDLL functionality in Deviare-InProc does have the functionality to inject the DLL into an existing process. But in that case, the process might already have performed its creation of IDispatch implementing COM objects, so it is too late to get anything useful from hooking CoGetClassObject().
We solved that with a hack that works nicely in many cases, by having the injected DLL itself create an object known to implement IDispatch, and hoping its Invoke implementation is the same as that used by the interesting things we want to trace.
Here is a snippet of a sample VBScript file:
Set objExcel = CreateObject("Excel.application")
set objExcelBook = objExcel.Workbooks.Open(FullName)
objExcel.application.visible=false
objExcel.application.displayalerts=false
objExcelBook.SaveAs replace(FileName, actualFileName, prefix & actualFileName) & "csv", 23
objExcel.Application.Quit
objExcel.Quit
And here is the corresponding output from tracing cscript executing that file. (In an actual use case, no VBScript source would obviously be available to inspect directly.)
Process #10104 successfully launched with dll injected!
Microsoft (R) Windows Script Host Version 5.812
Copyright (C) Microsoft Corporation. All rights reserved.
# CoGetClassObject({00024500-0000-0000-C000-000000000046}) (Excel.Application.15)
# riid={00000001-0000-0000-C000-000000000046}
# CoCreateInstance({0000032A-0000-0000-C000-000000000046}) (unknown)
# riid={00000149-0000-0000-C000-000000000046}
# result:95c668
# CoCreateInstance({00000339-0000-0000-C000-000000000046}) (unknown)
# riid={00000003-0000-0000-C000-000000000046}
# result:98aad8
# result:95dd8c
# Hooked Invoke 0 of 95de1c (old: 487001d) (orig: 76bafec0)
95de1c:Workbooks() -> IDispatch:98ed74
98ed74:Open({"c:\temp\b1.xls"}) : ({"c:\temp\b1.xls"}) -> IDispatch:98ea14
95de1c:Application() -> IDispatch:95de1c
95de1c:putVisible(FALSE)
95de1c:Application() -> IDispatch:95de1c
95de1c:putDisplayAlerts(FALSE)
98ea14:SaveAs(23,"c:\temp\converted_b1.csv")
95de1c:Application() -> IDispatch:95de1c
95de1c:Quit()
95de1c:Quit()
Our work on top of Deviare-InProc is available at https://github.com/CollaboraOnline/Deviare-InProc.
Binaries are available at https://people.collabora.com/~tml/injectdll/injectdll.zip (for 32-bit applications) and https://people.collabora.com/~tml/injectdll/injectdll64.zip (64-bit). The zip archive contains an executable, injectdll.exe (injectdll64.exe in the 64-bit case) and a DLL.
Unpack the zip archive somewhere. Then go there in Command Prompt, and in case the program you want to trace the IDispatch::Invoke use of is something you know how to start from the command line, you can enter this command:
injectdll.exe x:\path\to\program.exe “program arg1 arg2 …”
where program.exe is the executable you want to run, and arg1 arg2 … are command-line parameters it takes, if any.
If program.exe is a 64-bit program, instead download injectdll64.zip, and run injectdll64.exe, otherwise similar.
if you can’t start the program you want to investigate from the command line, but you need to inspect it after it has already started, just pass only the process id of the program to injectdll.exe instead. (Or injectdll64.exe) This is somewhat less likely to succeed, depending on how the program uses IDispatch.
In any case, the output (symbolic trace) will go to the standard output of the program being traced, which typically is nowhere at all, and not useful. It will not go to the standard output of the injectdll.exe program.
In order to redirect the output to a file, set an environment variable DEVIARE_LOGFILE that contains the full pathname to the log file to produce. This environment variable must be visible in the program that is being traced; it is not enough to set it in the Command Prompt window where you run injectdll.exe.
Obviously all this is a work in progress, and as needed will be hacked on further. For instance, the name "injectdll" is just the name of the original sample program in upstream Deviare-InProc; we should really rename it to something specific for this use case.
Binaries are available at https://people.collabora.com/~tml/injectdll/injectdll.zip (for 32-bit applications) and https://people.collabora.com/~tml/injectdll/injectdll64.zip (64-bit). The zip archive contains an executable, injectdll.exe (injectdll64.exe in the 64-bit case) and a DLL.
Unpack the zip archive somewhere. Then go there in Command Prompt, and in case the program you want to trace the IDispatch::Invoke use of is something you know how to start from the command line, you can enter this command:
injectdll.exe x:\path\to\program.exe “program arg1 arg2 …”
where program.exe is the executable you want to run, and arg1 arg2 … are command-line parameters it takes, if any.
If program.exe is a 64-bit program, instead download injectdll64.zip, and run injectdll64.exe, otherwise similar.
if you can’t start the program you want to investigate from the command line, but you need to inspect it after it has already started, just pass only the process id of the program to injectdll.exe instead. (Or injectdll64.exe) This is somewhat less likely to succeed, depending on how the program uses IDispatch.
In any case, the output (symbolic trace) will go to the standard output of the program being traced, which typically is nowhere at all, and not useful. It will not go to the standard output of the injectdll.exe program.
In order to redirect the output to a file, set an environment variable DEVIARE_LOGFILE that contains the full pathname to the log file to produce. This environment variable must be visible in the program that is being traced; it is not enough to set it in the Command Prompt window where you run injectdll.exe.
Obviously all this is a work in progress, and as needed will be hacked on further. For instance, the name "injectdll" is just the name of the original sample program in upstream Deviare-InProc; we should really rename it to something specific for this use case.
2015-04-03
What DAW (or other music software) is the right for me?
Dear lazyweb,
Is Ableton Live the right DAW for me?
I am not a musician or "producer". I don't know how to play any instrument. I don't really know musical notation (and have little desire to learn). But I do have some basic understanding of musical theory, an open mind, and would enjoy making experimental music. The kind I like to listen to. (Or, for the matter, why not more traditional electronic music, even dancey stuff, too.)
(I do listen to more "normal" music, too, not just experimental bleeps and drones;)
Among my favourite composers / artists are the usual suspects like Scelsi, Ligeti, Reich, Glass, Eno, Fripp, Kraftwerk, and contemporary ones like Max Richter, Anna Thorvaldsdottir, Nils Frahm, and of course acts like Circlesquare, Monolake, Plastikman etc. My favourite radio show is WNYC's New Sounds .
The software I have been ogling most is Ableton Live. What I like about it is that it is popular, cool, and has a thriving development, apparently, with relatively frequent updates etc. It seems not likely to go away suddenly. I love the total (?) lack of skeuomorphism in the user interface. (I strongly dislike the trend of faux real hardware look and feel in 3rd-party plugins.)
I certainly have no wish to use the "live" aspect of Live. And, as one of the main points, as I understand it, of Live is to make it easy to launch clips that will be automatically properly aligned in live performance scenarios, will Live be suitable for stuff like having multiple loops or patterns playing simultaneously *without* being synchronised? You know, like emulating Frippertronics, or patterns being slightly out of phase in the style of Reich, or Eno.
And what about microtonal aspects?
So, will Live have too many limitations? Should I look somewhere else? Maybe the generative music scene is what I should be looking into, like Intermorphic's Noatikl? Although with that I would definitely be afraid of the proprietary-software-maker-goes-belly-up scenario.
Maybe even some Open Source software? Csound? Pd? Note that I wouldn't want to get lured into hacking (programming) on some Open Source software, for once I want to be "just a user"...
Oh, and I use a Mac as my desktop environment, so that is also a limiting factor.
Or am I silly to even think I could create something interesting (to myself, that is) without starting by learning how to do stuff "by the book" first?
--tml
Is Ableton Live the right DAW for me?
I am not a musician or "producer". I don't know how to play any instrument. I don't really know musical notation (and have little desire to learn). But I do have some basic understanding of musical theory, an open mind, and would enjoy making experimental music. The kind I like to listen to. (Or, for the matter, why not more traditional electronic music, even dancey stuff, too.)
(I do listen to more "normal" music, too, not just experimental bleeps and drones;)
Among my favourite composers / artists are the usual suspects like Scelsi, Ligeti, Reich, Glass, Eno, Fripp, Kraftwerk, and contemporary ones like Max Richter, Anna Thorvaldsdottir, Nils Frahm, and of course acts like Circlesquare, Monolake, Plastikman etc. My favourite radio show is WNYC's New Sounds .
The software I have been ogling most is Ableton Live. What I like about it is that it is popular, cool, and has a thriving development, apparently, with relatively frequent updates etc. It seems not likely to go away suddenly. I love the total (?) lack of skeuomorphism in the user interface. (I strongly dislike the trend of faux real hardware look and feel in 3rd-party plugins.)
I certainly have no wish to use the "live" aspect of Live. And, as one of the main points, as I understand it, of Live is to make it easy to launch clips that will be automatically properly aligned in live performance scenarios, will Live be suitable for stuff like having multiple loops or patterns playing simultaneously *without* being synchronised? You know, like emulating Frippertronics, or patterns being slightly out of phase in the style of Reich, or Eno.
And what about microtonal aspects?
So, will Live have too many limitations? Should I look somewhere else? Maybe the generative music scene is what I should be looking into, like Intermorphic's Noatikl? Although with that I would definitely be afraid of the proprietary-software-maker-goes-belly-up scenario.
Maybe even some Open Source software? Csound? Pd? Note that I wouldn't want to get lured into hacking (programming) on some Open Source software, for once I want to be "just a user"...
Oh, and I use a Mac as my desktop environment, so that is also a limiting factor.
Or am I silly to even think I could create something interesting (to myself, that is) without starting by learning how to do stuff "by the book" first?
--tml
2015-02-14
2014-02-20
How can one catch OS X sandbox violations in a debugger?
Dear lazyweb,
Do you know of any way to catch OS X sandbox violations as they happen (in a huge program like LibreOffice where you don't have a clue what is going on in all places in the codebase) in a debugger (either gdb on the command line, or lldb under Xcode)? The trick using Dtrace in https://devforums.apple.com/message/874181#874181 does not seem to work in 10.9.1 at least.
Do you know of any way to catch OS X sandbox violations as they happen (in a huge program like LibreOffice where you don't have a clue what is going on in all places in the codebase) in a debugger (either gdb on the command line, or lldb under Xcode)? The trick using Dtrace in https://devforums.apple.com/message/874181#874181 does not seem to work in 10.9.1 at least.
2013-05-31
You can't win an argument against conspiracy theorists
Whatever you say, or don't say, will be taken as proof. If something is done, or is not done, in both cases that is a proof of the conspiracy.
An example, from the open source world: Mentioning a competitor project from which some of your code comes will be seen as proof that you unfairly want their fabulous reputation to rub over onto your project. On the other hand, not mentioning it will be seen as proof that you don't want people to know where your good stuff comes from.
2013-05-24
Another marketing achievement in LibreOffice: $(OUTDIR)/inc is gone
Thanks to the great work of our marketing experts like dtardon and mst__, no headers are any more copied ("delivered"), not even from 3rd-party packages, and thus $(OUTDIR)/inc is no more. Bye bye and good riddance! This makes the build tree again a bit cleaner, reduces build time somewhat, and can in cases avoid unneeded re-compilation of some source files.
2013-04-09
64-bit LibreOffice for OS X
During the ongoing SUSE Hack Week I had the possibility to make the CoreText-using code in LibreOffice work well enough to be usable. Thus a 64-bit build of LibreOffice is now also fairly usable, at least as long as you don't use any exotic functionality, knock on wood.
As such, it has been possible for a long time to build LibreOffice for OS X with a current Xcode 4.x and its Clang compiler, also as 64-bit code.
LibreOffice has traditionally used the ATSUI API to display text on Mac OS X. ATSUI was deprecated years ago and superseded by CoreText. ATSUI is still present in the OS, but only for 32-bit code. 64-bit code has to use CoreText. Which is a good thing, of course; it forces developers to finally stop using obsolete API if they want to produce 64-bit code.
Naturally, CoreText is also what iOS provides, so it was necessary to get the CoreText engine into shape also for the benefit of the iOS porting effort.
Note that this work has been available in the LibreOffice source code repository continuously, even while known to be far from usable. I don't believe in the "check in the code when it is ready" way of working in an Open Source community.
There are still some known glitches. For instance, the placement of glyphs on a line that you are editing is slightly "nervous". Glyphs might jump back and forth a pixel when you add or remove characters. I will work on fixing this.
Those who are interested can find daily builds. ("Daily" should be taken with a grain of salt; whether builds have been uploaded for a given day or not depends on whether I remember to keep my tinderbox running, and in what shape the master branch is.)
If you want to build a CoreText-using LibreOffice yourself, use the --enable-coretext option. To build a 64-bit LibreOffice, use --enable-64-bit.
Thanks to Norbert Thiebaud for his initial work on using CoreText.
As such, it has been possible for a long time to build LibreOffice for OS X with a current Xcode 4.x and its Clang compiler, also as 64-bit code.
LibreOffice has traditionally used the ATSUI API to display text on Mac OS X. ATSUI was deprecated years ago and superseded by CoreText. ATSUI is still present in the OS, but only for 32-bit code. 64-bit code has to use CoreText. Which is a good thing, of course; it forces developers to finally stop using obsolete API if they want to produce 64-bit code.
Naturally, CoreText is also what iOS provides, so it was necessary to get the CoreText engine into shape also for the benefit of the iOS porting effort.
Note that this work has been available in the LibreOffice source code repository continuously, even while known to be far from usable. I don't believe in the "check in the code when it is ready" way of working in an Open Source community.
There are still some known glitches. For instance, the placement of glyphs on a line that you are editing is slightly "nervous". Glyphs might jump back and forth a pixel when you add or remove characters. I will work on fixing this.
Those who are interested can find daily builds. ("Daily" should be taken with a grain of salt; whether builds have been uploaded for a given day or not depends on whether I remember to keep my tinderbox running, and in what shape the master branch is.)
If you want to build a CoreText-using LibreOffice yourself, use the --enable-coretext option. To build a 64-bit LibreOffice, use --enable-64-bit.
Thanks to Norbert Thiebaud for his initial work on using CoreText.
