Hungry Mind , Blog about everything in IT - C#, Java, C++, .NET, Windows, WinAPI, ...

How to prevent MFC dialog closing on Enter and Escape keys

In this article I explain how to prevent an MFC dialog from handling the Enter and Escape keys and not pass it on. With all required details for you to completely understand why does it work the way it does. There are several possible solutions, so you can choose the most suitable one.

The problem is the following: you have a dialog, no matter - top level window or just a child container; you press Escape or Enter keys - the dialog disappears, even if there are no OK and Cancel buttons on the dialog. It's very funny seeing a child dialog drops out it's parent.

Let's see why this happens:

OrderXMLReaderShell.exe!CFilterEditorDlgBase::OnCancel()  Line 61 C++
  mfc100ud.dll!_AfxDispatchCmdMsg(CCmdTarget * pTarget, unsigned int nID, int nCode, void (void)* pfn, void * pExtra, unsigned int nSig, AFX_CMDHANDLERINFO * pHandlerInfo)  Line 82 C++
  mfc100ud.dll!CCmdTarget::OnCmdMsg(unsigned int nID, int nCode, void * pExtra, AFX_CMDHANDLERINFO * pHandlerInfo)  Line 381 + 0x27 bytes C++
  mfc100ud.dll!CDialog::OnCmdMsg(unsigned int nID, int nCode, void * pExtra, AFX_CMDHANDLERINFO * pHandlerInfo)  Line 87 + 0x18 bytes C++
  mfc100ud.dll!CWnd::OnCommand(unsigned int wParam, long lParam)  Line 2729 C++
  mfc100ud.dll!CWnd::OnWndMsg(unsigned int message, unsigned int wParam, long lParam, long * pResult)  Line 2101 + 0x1e bytes C++
  mfc100ud.dll!CWnd::WindowProc(unsigned int message, unsigned int wParam, long lParam)  Line 2087 + 0x20 bytes C++
  mfc100ud.dll!AfxCallWndProc(CWnd * pWnd, HWND__ * hWnd, unsigned int nMsg, unsigned int wParam, long lParam)  Line 257 + 0x1c bytes C++
  mfc100ud.dll!AfxWndProc(HWND__ * hWnd, unsigned int nMsg, unsigned int wParam, long lParam)  Line 420 C++
  mfc100ud.dll!AfxWndProcBase(HWND__ * hWnd, unsigned int nMsg, unsigned int wParam, long lParam)  Line 420 + 0x15 bytes C++
  user32.dll!_InternalCallWinProc@20()  + 0x23 bytes 
  user32.dll!_UserCallWinProcCheckWow@32()  + 0xb3 bytes 
  user32.dll!_SendMessageWorker@20()  + 0xee bytes 
  user32.dll!_SendMessageW@16()  + 0x49 bytes 
  user32.dll!_IsDialogMessageW@8()  + 0xef46 bytes 
> mfc100ud.dll!CWnd::IsDialogMessageW(tagMSG * lpMsg)  Line 198 C++
  mfc100ud.dll!CWnd::PreTranslateInput(tagMSG * lpMsg)  Line 4713 C++
  mfc100ud.dll!CDialog::PreTranslateMessage(tagMSG * pMsg)  Line 82 C++
  OrderXMLReaderShell.exe!CFindDlg::PreTranslateMessage(tagMSG * pMsg)  Line 56 C++
  mfc100ud.dll!CWnd::WalkPreTranslateTree(HWND__ * hWndStop, tagMSG * pMsg)  Line 3311 + 0x14 bytes C++
  mfc100ud.dll!AfxInternalPreTranslateMessage(tagMSG * pMsg)  Line 233 + 0x12 bytes C++
  mfc100ud.dll!CWinThread::PreTranslateMessage(tagMSG * pMsg)  Line 777 + 0x9 bytes C++
  mfc100ud.dll!AfxPreTranslateMessage(tagMSG * pMsg)  Line 252 + 0x11 bytes C++
  mfc100ud.dll!AfxInternalPumpMessage()  Line 178 + 0x18 bytes C++
  mfc100ud.dll!CWinThread::PumpMessage()  Line 900 C++
  mfc100ud.dll!CWinThread::Run()  Line 629 + 0xd bytes C++
  mfc100ud.dll!CWinApp::Run()  Line 832 C++
  mfc100ud.dll!AfxWinMain(HINSTANCE__ * hInstance, HINSTANCE__ * hPrevInstance, wchar_t * lpCmdLine, int nCmdShow)  Line 47 + 0xd bytes C++
  OrderXMLReaderShell.exe!wWinMain(HINSTANCE__ * hInstance, HINSTANCE__ * hPrevInstance, wchar_t * lpCmdLine, int nCmdShow)  Line 26 C++
  OrderXMLReaderShell.exe!__tmainCRTStartup()  Line 547 + 0x2c bytes C
  OrderXMLReaderShell.exe!wWinMainCRTStartup()  Line 371 C
  kernel32.dll!@BaseThreadInitThunk@12()  + 0x12 bytes 
  ntdll.dll!___RtlUserThreadStart@8()  + 0x27 bytes 
  ntdll.dll!__RtlUserThreadStart@8()  + 0x1b bytes 

The execution flow got into CDialog::OnCancel virtual method. Here is the default MFC's implementation:

void CDialog::OnCancel()

It simply ends the dialog, thus destroying the window.

Lets find out how do we get there. The control flow is the following:

  1. Escape key is down
  2. WM_KEYDOWN message is queued into message queue
  3. AfxInternalPumpMessage gets the message from the queue and invokes AfxPreTranslateMessage
  4. CWnd::IsDialogMessage is reached
  5. IsDialogMessage WinAPI function is called by CWnd::IsDialogMessage; this function is the core of dialog management - it does things like keyboard navigation.
  6. There is a user32.dll!_SendMessageW@16() call from IsDialogMessageW
  7. Which brings us to CWnd::OnCommand with wParam equals to IDCANCEL
  8. End finally to CDialog::OnCancel

IsDialogMessageW examines the message and invokes SendMessage(hwndDlg, WM_COMMAND, MAKELONG(IDCANCEL, BN_CLICKED), ...);.

Short summary - MFC calls IsDialogMessage API function, which in turns emulates IDCANCEL button click, which is being handled by MFC eventually destroying the dialog.

It is not obvious why IsDialogMessageW emulates IDOK and IDCANCEL button clicks. I believe this is done to make developer's life easier. Imagine you have no control over the message loop, can't do any message pre translation. You'll have to handle Enter and Escape keys for every control on the dialog. That's pain.

What can we do to solve the problem? You can't change OS behavior for sure, but can do the following:

  1. Override CDialog::OnOK/OnCancel and do nothing

    The following applies:

    • Dialog is not destroyed on Enter or Escape
    • Focused inline controls don't receive WM_KEYDOWN for VK_ENTER and VK_ESCAPE codes
    • OK/Cancel button clicks have no effect
  2. Override CDialog::PreTranslateMessage and don't call the base if pMsg->message == WM_KEYDOWN && (pMsg->wParam == VK_ESCAPE || pMsg->wParam == VK_RETURN)

    If CDialog::PreTranslateMessage returns TRUE the following applies:

    • Dialog is not destroyed on Enter or Escape
    • Focused inline controls don't receive WM_KEYDOWN for VK_RETURN and VK_ESCAPE codes, unless you do dispatch them before CDialog::PreTranslateMessage returns
    • OK/Cancel button clicks have desired effect

    If CDialog::PreTranslateMessage returns FALSE the following applies:

    • MFC calls parent's PreTranslateMessage (which brings us back if the parent is a CDialog instance)
  3. Override CWinThread::PreTranslateMessage, examine the message and don't call the base

    The following applies:

    • CWnd::WalkPreTranslateTree is not invoked, thus disabling message pre translation along with IsDialogMessage call (specific messages only, it's not as scary as it seems, but hits the performance and hard to implement properly)

      If CWinThread::PreTranslateMessage returns TRUE the following applies:

      • Message is not dispatched
      • Focused inline controls don't receive WM_KEYDOWN for VK_RETURN and VK_ESCAPE codes

      If CWinThread::PreTranslateMessage returns FALSE the following applies:

      • Message is dispatched
      • Focused inline controls receive WM_KEYDOWN for VK_RETURN and VK_ESCAPE codes
    • The behavior is application wide

The very best option would be to override CWnd::IsDialogMessage and return in case of VK_RETURN and VK_ESCAPE, but this is impossible since CWnd::IsDialogMessage is not virtual. CWnd::IsDialogMessage is an API function wrapper, the closest item to modify the behavior.

So which method is the best? Each does the trick, but... Option 3 has serious drawback. Option 1 and 2 are class specific. I vote for option 2. You could create some base CDialog class with the trick and use it for all your dialogs.

13 коммент.:

Анонимный комментирует...

This post will assist the internet users for creating new weblog or even a blog from start to end.
Here is my webpage : GFI Norte

Анонимный комментирует...

Valuable information. Lucky me I found your website by accident, and
I'm stunned why this coincidence didn't came about in advance!
I bookmarked it.

Feel free to surf to my page - insomnia uvu 2012
My blog post ; insomnia 9dpo

Анонимный комментирует...

Thank you for the good writeup. It in fact was a amusement account it.

Look advanced to more added agreeable from you! However, how could we communicate?

Also visit my web site - insomnia 8th street
Feel free to surf my site ... insomnia 1960

Анонимный комментирует...

Wow that was strange. I just wrote an really long comment but after I clicked submit my comment didn't show up. Grrrr... well I'm not writing all that over again.
Anyway, just wanted to say superb blog!

Take a look at my homepage - online backup review
Here is my web page - online backup reviews

Анонимный комментирует...

Hi, i think that i saw you visited my blog thus
i came to “return the favor”.I am trying
to find things to enhance my website!I suppose its ok to use a few
of your ideas!!

Here is my weblog - insomnia quit drinking
Also visit my web page : insomnia and pms

Анонимный комментирует...

We are a bunch of volunteers and starting a new scheme in our community.
Your site provided us with helpful info to work on. You have performed a formidable process and our
entire neighborhood can be grateful to you.

my web site - iphone 5 nano

Анонимный комментирует...

I every timе spent my half аn hоur to read this
ωeblog's articles daily along with a cup of coffee.

Here is my blog post - diseño web

Анонимный комментирует...

What i do not undeгstooԁ is in reality how yоu're not actually much more smartly-appreciated than you might be now. You are very intelligent. You already know thus considerably relating to this subject, made me in my view imagine it from a lot of various angles. Its like men and women aren't involved untіl it is something
to accomplish wіth Woman gagа! Your individual ѕtuffѕ great.

All the time deal with іt up!

Feel fгeе to surf to mу homеpage :
: Création boutique en ligne

Анонимный комментирует...

Ηi thеre to еveгу ѕingle one, іt's genuinely a good for me to pay a visit this web page, it contains priceless Information.

Stop by my weblog :: desarrollo web

Анонимный комментирует...

Have you ever thought about аdding a lіttle bіt mоre than just уour аrticles?
I meаn, what you say is іmportant anԁ
all. Нοweνer imagine if you added
some great ѵisualѕ or video clірs to give уour
posts more, "pop"! Your cοntent iѕ excellent but with pics and vіdеo clipѕ, this website
could certaіnly be one of the mоst benеficіal in
its field. Wondeгful blog!

Also viѕit my homepage:

Анонимный комментирует...

Wonԁerful aгticlе! We aгe linking to this grеat
post on our site. Κеep up the greаt writing.

my page Création boutique en ligne

Анонимный комментирует...

I read this paragraph fully about the resemblance of hottest and earlier technologies, it's amazing article.

Feel free to visit my webpage thande pas cher

Анонимный комментирует...

Hiya! Quick question that's completely off topic. Do you know how to make your site mobile friendly? My blog looks weird when viewing from my iphone4. I'm trying to find a theme or plugin that might be able to fix this issue.

If you have any recommendations, please share.

Here is my blog post: lavarse

Отправить комментарий

Copyright 2007-2011 Chabster