tag:blogger.com,1999:blog-2646543030379124052024-03-18T11:47:37.832+02:00Hungry MindBlog about everything in IT - C#, Java, C++, .NET, Windows, WinAPI, ...Anonymoushttp://www.blogger.com/profile/14003906116514303118noreply@blogger.comBlogger363125tag:blogger.com,1999:blog-264654303037912405.post-70026576082481290432016-02-11T08:00:00.002+02:002016-02-11T08:00:42.496+02:00How to compute IP and TCP checksum<p>
Whenever I need to code something, the very first thing I do is google for a ready to use solution. Some time ago I had to calculate IP and TCP checksums. Every peace of code I googled was unreadable junk. I wonder is it really hard to write good looking readable code, so you can just copy and paste it?!
</p>
<pre style="background: white; font-family: Consolas; font-size: 15px;"><span style="color: orangered;">#pragma</span> <span style="color: orangered;">pack</span><span style="color: #800040;">(</span><span style="color: orangered;">push</span><span style="color: #800040;">,</span> <span style="color: darkgreen;">1</span><span style="color: #800040;">)</span>
<span style="color: navy;">struct</span> <span style="color: #892be0;">IPV4_HDR</span>
<span style="color: #800040;">{</span>
<span style="color: #892be0;">uint8_t</span> <span style="color: #20b08a;">ihl</span> <span style="color: #800040;">:</span> <span style="color: darkgreen;">4</span><span style="color: #800040;">;</span>
<span style="color: #892be0;">uint8_t</span> <span style="color: #20b08a;">version</span> <span style="color: #800040;">:</span> <span style="color: darkgreen;">4</span><span style="color: #800040;">;</span>
<span style="color: #892be0;">uint8_t</span> <span style="color: #20b08a;">tos</span><span style="color: #800040;">;</span>
<span style="color: #892be0;">uint16_t</span> <span style="color: #20b08a;">total_length</span><span style="color: #800040;">;</span>
<span style="color: #892be0;">uint16_t</span> <span style="color: #20b08a;">identification</span><span style="color: #800040;">;</span>
<span style="color: #892be0;">uint16_t</span> <span style="color: #20b08a;">fragment_offset</span> <span style="color: #800040;">:</span> <span style="color: darkgreen;">13</span><span style="color: #800040;">;</span>
<span style="color: #892be0;">uint16_t</span> <span style="color: #20b08a;">more_fragment</span> <span style="color: #800040;">:</span> <span style="color: darkgreen;">1</span><span style="color: #800040;">;</span>
<span style="color: #892be0;">uint16_t</span> <span style="color: #20b08a;">dont_fragment</span> <span style="color: #800040;">:</span> <span style="color: darkgreen;">1</span><span style="color: #800040;">;</span>
<span style="color: #892be0;">uint16_t</span> <span style="color: #20b08a;">reserved_zero</span> <span style="color: #800040;">:</span> <span style="color: darkgreen;">1</span><span style="color: #800040;">;</span>
<span style="color: #892be0;">uint8_t</span> <span style="color: #20b08a;">ttl</span><span style="color: #800040;">;</span>
<span style="color: #892be0;">uint8_t</span> <span style="color: #20b08a;">proto</span><span style="color: #800040;">;</span>
<span style="color: #892be0;">uint16_t</span> <span style="color: #20b08a;">header_checksum</span><span style="color: #800040;">;</span>
<span style="color: #892be0;">uint32_t</span> <span style="color: #20b08a;">src_ip</span><span style="color: #800040;">;</span>
<span style="color: #892be0;">uint32_t</span> <span style="color: #20b08a;">dst_ip</span><span style="color: #800040;">;</span>
<span style="color: #800040;">};</span>
<span style="color: navy;">struct</span> <span style="color: #892be0;">TCP_HDR</span>
<span style="color: #800040;">{</span>
<span style="color: #892be0;">uint16_t</span> <span style="color: #20b08a;">src_port</span><span style="color: #800040;">;</span>
<span style="color: #892be0;">uint16_t</span> <span style="color: #20b08a;">dst_port</span><span style="color: #800040;">;</span>
<span style="color: #892be0;">uint32_t</span> <span style="color: #20b08a;">seqn</span><span style="color: #800040;">;</span>
<span style="color: #892be0;">uint32_t</span> <span style="color: #20b08a;">ackn</span><span style="color: #800040;">;</span>
<span style="color: #892be0;">uint8_t</span> <span style="color: #20b08a;">ns</span> <span style="color: #800040;">:</span> <span style="color: darkgreen;">1</span><span style="color: #800040;">;</span>
<span style="color: #892be0;">uint8_t</span> <span style="color: #20b08a;">reserved</span> <span style="color: #800040;">:</span> <span style="color: darkgreen;">3</span><span style="color: #800040;">;</span>
<span style="color: #892be0;">uint8_t</span> <span style="color: #20b08a;">data_offset</span> <span style="color: #800040;">:</span> <span style="color: darkgreen;">4</span><span style="color: #800040;">;</span>
<span style="color: #892be0;">uint8_t</span> <span style="color: #20b08a;">fin</span> <span style="color: #800040;">:</span> <span style="color: darkgreen;">1</span><span style="color: #800040;">;</span>
<span style="color: #892be0;">uint8_t</span> <span style="color: #20b08a;">syn</span> <span style="color: #800040;">:</span> <span style="color: darkgreen;">1</span><span style="color: #800040;">;</span>
<span style="color: #892be0;">uint8_t</span> <span style="color: #20b08a;">rst</span> <span style="color: #800040;">:</span> <span style="color: darkgreen;">1</span><span style="color: #800040;">;</span>
<span style="color: #892be0;">uint8_t</span> <span style="color: #20b08a;">psh</span> <span style="color: #800040;">:</span> <span style="color: darkgreen;">1</span><span style="color: #800040;">;</span>
<span style="color: #892be0;">uint8_t</span> <span style="color: #20b08a;">ack</span> <span style="color: #800040;">:</span> <span style="color: darkgreen;">1</span><span style="color: #800040;">;</span>
<span style="color: #892be0;">uint8_t</span> <span style="color: #20b08a;">urg</span> <span style="color: #800040;">:</span> <span style="color: darkgreen;">1</span><span style="color: #800040;">;</span>
<span style="color: #892be0;">uint8_t</span> <span style="color: #20b08a;">ecn</span> <span style="color: #800040;">:</span> <span style="color: darkgreen;">1</span><span style="color: #800040;">;</span>
<span style="color: #892be0;">uint8_t</span> <span style="color: #20b08a;">cwr</span> <span style="color: #800040;">:</span> <span style="color: darkgreen;">1</span><span style="color: #800040;">;</span>
<span style="color: #892be0;">uint16_t</span> <span style="color: #20b08a;">window</span><span style="color: #800040;">;</span>
<span style="color: #892be0;">uint16_t</span> <span style="color: #20b08a;">checksum</span><span style="color: #800040;">;</span>
<span style="color: #892be0;">uint16_t</span> <span style="color: #20b08a;">urgent_pointer</span><span style="color: #800040;">;</span>
<span style="color: #800040;">};</span>
<span style="color: orangered;">#pragma</span> <span style="color: orangered;">pack</span><span style="color: #800040;">(</span><span style="color: orangered;">pop</span><span style="color: #800040;">)</span>
<span style="color: #892be0;">uint16_t</span> <span style="color: crimson;">compute_checksum</span><span style="color: #800040;">(</span><span style="color: #892be0;">uint16_t</span> <span style="color: navy;">const</span> <span style="color: #800040;">*</span><span style="color: saddlebrown;">data</span><span style="color: #800040;">,</span> <span style="color: #892be0;">size_t</span> <span style="color: saddlebrown;">count</span><span style="color: #800040;">)</span>
<span style="color: #800040;">{</span>
<span style="color: #892be0;">uint32_t</span> <span style="color: lightslategrey;">sum</span> <span style="color: #800040;">=</span> <span style="color: darkgreen;">0</span><span style="color: #800040;">;</span>
<span style="color: navy;">while</span> <span style="color: #800040;">(</span><span style="color: saddlebrown;">count</span> <span style="color: #800040;">></span> <span style="color: darkgreen;">1</span><span style="color: #800040;">)</span> <span style="color: #800040;">{</span>
<span style="color: lightslategrey;">sum</span> <span style="color: #800040;">+=</span> <span style="color: #800040;">*</span><span style="color: saddlebrown;">data</span><span style="color: #800040;">++;</span>
<span style="color: saddlebrown;">count</span> <span style="color: #800040;">-=</span> <span style="color: navy;">sizeof</span><span style="color: #800040;">(</span><span style="color: #892be0;">uint16_t</span><span style="color: #800040;">);</span>
<span style="color: #800040;">}</span>
<span style="color: navy;">if</span> <span style="color: #800040;">(</span><span style="color: saddlebrown;">count</span> <span style="color: #800040;">></span> <span style="color: darkgreen;">0</span><span style="color: #800040;">)</span> <span style="color: lightslategrey;">sum</span> <span style="color: #800040;">+=</span> <span style="color: #800040;">((*</span><span style="color: saddlebrown;">data</span><span style="color: #800040;">)</span> <span style="color: #800040;">&</span> <span style="color: crimson;">htons</span><span style="color: #800040;">(</span><span style="color: darkgreen;">0xFF00</span><span style="color: #800040;">));</span>
<span style="color: navy;">while</span> <span style="color: #800040;">(</span><span style="color: lightslategrey;">sum</span> <span style="color: #800040;">>></span> <span style="color: darkgreen;">16</span><span style="color: #800040;">)</span> <span style="color: lightslategrey;">sum</span> <span style="color: #800040;">=</span> <span style="color: #800040;">(</span><span style="color: lightslategrey;">sum</span> <span style="color: #800040;">&</span> <span style="color: darkgreen;">0xFFFF</span><span style="color: #800040;">)</span> <span style="color: #800040;">+</span> <span style="color: #800040;">(</span><span style="color: lightslategrey;">sum</span> <span style="color: #800040;">>></span> <span style="color: darkgreen;">16</span><span style="color: #800040;">);</span>
<span style="color: lightslategrey;">sum</span> <span style="color: #800040;">=</span> <span style="color: #800040;">~</span><span style="color: lightslategrey;">sum</span><span style="color: #800040;">;</span>
<span style="color: navy;">return</span> <span style="color: navy;">static_cast</span><span style="color: #800040;"><</span><span style="color: #892be0;">uint16_t</span><span style="color: #800040;">>(</span><span style="color: lightslategrey;">sum</span><span style="color: #800040;">);</span>
<span style="color: #800040;">}</span>
<span style="color: #892be0;">uint16_t</span> <span style="color: crimson;">compute_ip_checksum</span><span style="color: #800040;">(</span><span style="color: #892be0;">IPV4_HDR</span> <span style="color: #800040;">*</span><span style="color: saddlebrown;">ip</span><span style="color: #800040;">)</span>
<span style="color: #800040;">{</span>
<span style="color: saddlebrown;">ip</span><span style="color: #800040;">-></span><span style="color: #20b08a;">header_checksum</span> <span style="color: #800040;">=</span> <span style="color: darkgreen;">0U</span><span style="color: #800040;">;</span>
<span style="color: navy;">return</span> <span style="color: #800040;">(</span><span style="color: saddlebrown;">ip</span><span style="color: #800040;">-></span><span style="color: #20b08a;">header_checksum</span> <span style="color: #800040;">=</span> <span style="color: crimson;">compute_checksum</span><span style="color: #800040;">(</span><span style="color: navy;">reinterpret_cast</span><span style="color: #800040;"><</span><span style="color: #892be0;">uint16_t</span> <span style="color: navy;">const</span> <span style="color: #800040;">*>(</span><span style="color: saddlebrown;">ip</span><span style="color: #800040;">),</span> <span style="color: saddlebrown;">ip</span><span style="color: #800040;">-></span><span style="color: #20b08a;">ihl</span> <span style="color: #800040;"><<</span> <span style="color: darkgreen;">2</span><span style="color: #800040;">));</span>
<span style="color: #800040;">}</span>
<span style="color: #892be0;">uint16_t</span> <span style="color: crimson;">compute_tcp_checksum</span><span style="color: #800040;">(</span><span style="color: #892be0;">IPV4_HDR</span> <span style="color: navy;">const</span> <span style="color: #800040;">*</span><span style="color: saddlebrown;">ip</span><span style="color: #800040;">,</span> <span style="color: #892be0;">uint16_t</span> <span style="color: #800040;">*</span><span style="color: saddlebrown;">payload</span><span style="color: #800040;">)</span>
<span style="color: #800040;">{</span>
<span style="color: #892be0;">uint32_t</span> <span style="color: lightslategrey;">sum</span> <span style="color: #800040;">=</span> <span style="color: darkgreen;">0</span><span style="color: #800040;">;</span>
<span style="color: #892be0;">uint16_t</span> <span style="color: lightslategrey;">tcp_len</span> <span style="color: #800040;">=</span> <span style="color: crimson;">ntohs</span><span style="color: #800040;">(</span><span style="color: saddlebrown;">ip</span><span style="color: #800040;">-></span><span style="color: #20b08a;">total_length</span><span style="color: #800040;">)</span> <span style="color: #800040;">-</span> <span style="color: #800040;">(</span><span style="color: saddlebrown;">ip</span><span style="color: #800040;">-></span><span style="color: #20b08a;">ihl</span> <span style="color: #800040;"><<</span> <span style="color: darkgreen;">2</span><span style="color: #800040;">);</span>
<span style="color: navy;">auto</span> <span style="color: navy;">const</span> <span style="color: lightslategrey;">tcp</span> <span style="color: #800040;">=</span> <span style="color: navy;">reinterpret_cast</span><span style="color: #800040;"><</span><span style="color: #892be0;">TCP_HDR</span> <span style="color: #800040;">*>(</span><span style="color: saddlebrown;">payload</span><span style="color: #800040;">);</span>
<span style="color: lightslategrey;">sum</span> <span style="color: #800040;">+=</span> <span style="color: #800040;">(</span><span style="color: saddlebrown;">ip</span><span style="color: #800040;">-></span><span style="color: #20b08a;">src_ip</span> <span style="color: #800040;">>></span> <span style="color: darkgreen;">16</span><span style="color: #800040;">)</span> <span style="color: #800040;">&</span> <span style="color: darkgreen;">0xFFFF</span><span style="color: #800040;">;</span>
<span style="color: lightslategrey;">sum</span> <span style="color: #800040;">+=</span> <span style="color: #800040;">(</span><span style="color: saddlebrown;">ip</span><span style="color: #800040;">-></span><span style="color: #20b08a;">src_ip</span><span style="color: #800040;">)</span> <span style="color: #800040;">&</span> <span style="color: darkgreen;">0xFFFF</span><span style="color: #800040;">;</span>
<span style="color: lightslategrey;">sum</span> <span style="color: #800040;">+=</span> <span style="color: #800040;">(</span><span style="color: saddlebrown;">ip</span><span style="color: #800040;">-></span><span style="color: #20b08a;">dst_ip</span> <span style="color: #800040;">>></span> <span style="color: darkgreen;">16</span><span style="color: #800040;">)</span> <span style="color: #800040;">&</span> <span style="color: darkgreen;">0xFFFF</span><span style="color: #800040;">;</span>
<span style="color: lightslategrey;">sum</span> <span style="color: #800040;">+=</span> <span style="color: #800040;">(</span><span style="color: saddlebrown;">ip</span><span style="color: #800040;">-></span><span style="color: #20b08a;">dst_ip</span><span style="color: #800040;">)</span> <span style="color: #800040;">&</span> <span style="color: darkgreen;">0xFFFF</span><span style="color: #800040;">;</span>
<span style="color: lightslategrey;">sum</span> <span style="color: #800040;">+=</span> <span style="color: crimson;">htons</span><span style="color: #800040;">(</span><span style="color: olive;">IPPROTO_TCP</span><span style="color: #800040;">);</span>
<span style="color: lightslategrey;">sum</span> <span style="color: #800040;">+=</span> <span style="color: crimson;">htons</span><span style="color: #800040;">(</span><span style="color: lightslategrey;">tcp_len</span><span style="color: #800040;">);</span>
<span style="color: lightslategrey;">tcp</span><span style="color: #800040;">-></span><span style="color: #20b08a;">checksum</span> <span style="color: #800040;">=</span> <span style="color: darkgreen;">0</span><span style="color: #800040;">;</span>
<span style="color: navy;">while</span> <span style="color: #800040;">(</span><span style="color: lightslategrey;">tcp_len</span> <span style="color: #800040;">></span> <span style="color: darkgreen;">1</span><span style="color: #800040;">)</span> <span style="color: #800040;">{</span>
<span style="color: lightslategrey;">sum</span> <span style="color: #800040;">+=</span> <span style="color: #800040;">*</span><span style="color: saddlebrown;">payload</span><span style="color: #800040;">++;</span>
<span style="color: lightslategrey;">tcp_len</span> <span style="color: #800040;">-=</span> <span style="color: navy;">sizeof</span><span style="color: #800040;">(</span><span style="color: #892be0;">uint16_t</span><span style="color: #800040;">);</span>
<span style="color: #800040;">}</span>
<span style="color: navy;">if</span> <span style="color: #800040;">(</span><span style="color: lightslategrey;">tcp_len</span> <span style="color: #800040;">></span> <span style="color: darkgreen;">0</span><span style="color: #800040;">)</span> <span style="color: lightslategrey;">sum</span> <span style="color: #800040;">+=</span> <span style="color: #800040;">((*</span><span style="color: saddlebrown;">payload</span><span style="color: #800040;">)</span> <span style="color: #800040;">&</span> <span style="color: crimson;">htons</span><span style="color: #800040;">(</span><span style="color: darkgreen;">0xFF00</span><span style="color: #800040;">));</span>
<span style="color: navy;">while</span> <span style="color: #800040;">(</span><span style="color: lightslategrey;">sum</span> <span style="color: #800040;">>></span> <span style="color: darkgreen;">16</span><span style="color: #800040;">)</span> <span style="color: lightslategrey;">sum</span> <span style="color: #800040;">=</span> <span style="color: #800040;">(</span><span style="color: lightslategrey;">sum</span> <span style="color: #800040;">&</span> <span style="color: darkgreen;">0xffff</span><span style="color: #800040;">)</span> <span style="color: #800040;">+</span> <span style="color: #800040;">(</span><span style="color: lightslategrey;">sum</span> <span style="color: #800040;">>></span> <span style="color: darkgreen;">16</span><span style="color: #800040;">);</span>
<span style="color: lightslategrey;">sum</span> <span style="color: #800040;">=</span> <span style="color: #800040;">~</span><span style="color: lightslategrey;">sum</span><span style="color: #800040;">;</span>
<span style="color: navy;">return</span> <span style="color: #800040;">(</span><span style="color: lightslategrey;">tcp</span><span style="color: #800040;">-></span><span style="color: #20b08a;">checksum</span> <span style="color: #800040;">=</span> <span style="color: navy;">static_cast</span><span style="color: #800040;"><</span><span style="color: #892be0;">uint16_t</span><span style="color: #800040;">>(</span><span style="color: lightslategrey;">sum</span><span style="color: #800040;">));</span>
<span style="color: #800040;">}</span>
</pre>
Anonymoushttp://www.blogger.com/profile/14003906116514303118noreply@blogger.com340tag:blogger.com,1999:blog-264654303037912405.post-83624539682184198932015-07-25T04:45:00.001+03:002015-07-28T02:00:23.525+03:00__await is still broken in Visual Studio 2015 RTM<p>
<a href="https://connect.microsoft.com/VisualStudio/feedback/details/1587599">Connect issue</a>
</p>
<p>
The following simple code does not work:
</p>
<pre class="listing code">
#include "stdafx.h"
#include <experimental\generator>
#include <future>
using namespace std;
using namespace std::experimental;
future<int> get_int()
{
return async([] { return 0; });
}
future<int> wait_int()
{
return __await get_int();
}
int main()
{
auto const i = wait_int().get();
return 0;
}
</pre>
<p>
App hits <code>int 3</code> instruction (XXX.exe has triggered a breakpoint) and then hangs forever waiting for suspended coroutine to finish. Bravo!
My investigation shows that compiler generates invalid instructions within XXX.exe!wait_int$_ResumeCoro$2() function:
</p>
<pre>
00007FF6166FB0F0 mov qword ptr [rsp+8],rcx
00007FF6166FB0F5 push rbp
00007FF6166FB0F6 sub rsp,30h
00007FF6166FB0FA mov qword ptr [rsp+20h],0FFFFFFFFFFFFFFFEh
00007FF6166FB103 mov rbp,qword ptr [$S2]
00007FF6166FB108 mov eax,dword ptr [rbp+20h]
00007FF6166FB10B mov dword ptr [rbp+78h],eax
00007FF6166FB10E cmp dword ptr [rbp+78h],5
</pre>
<p>
<code>rbp</code> has the address of coroutine frame, frame has two consequent 64-bit values - address of resume method and some internal state flag, which is set to 2 initially.
<code>dword ptr [rbp+20h]</code> - this instruction obtains the flag, but the offset is completely wrong, it must be <code>dword ptr [rbp+8h]</code>.
So all cases of state flag switch are bypassed. Default case makes the assert hit. Boom.
</p>
<p class="warning">
UPD: It doesn't support Debug Information Format == Program Database for Edit And Continue (/Zl). With this setting set to something else the code is generated properly: <code>mov eax, [rbp+8]</code>.
</p>Anonymoushttp://www.blogger.com/profile/14003906116514303118noreply@blogger.com5tag:blogger.com,1999:blog-264654303037912405.post-71896632284618788492015-07-03T08:43:00.000+03:002015-07-03T08:43:19.039+03:00Lindemann<p>Он бесподобен, я хочу от него детей!!!!!!11</p>
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/nS1ztWGwzgQ/0.jpg" frameborder="0" src="https://www.youtube.com/embed/nS1ztWGwzgQ?feature=player_embedded"></iframe>
Anonymoushttp://www.blogger.com/profile/14003906116514303118noreply@blogger.com2tag:blogger.com,1999:blog-264654303037912405.post-81989000960823694802015-01-30T21:27:00.000+02:002015-01-30T21:27:00.959+02:00Establishing an RDP connection with a Windows 8.1 client from Mac OS X<div dir="ltr" style="text-align: left;" trbidi="on">
<a href="http://kb.mit.edu/confluence/display/istcontrib/Establishing+an+RDP+connection+with+a+Windows+8.1+client+from+Mac+OS+X" target="_blank">http://kb.mit.edu/confluence/display/istcontrib/Establishing+an+RDP+connection+with+a+Windows+8.1+client+from+Mac+OS+X</a></div>
Anonymoushttp://www.blogger.com/profile/14003906116514303118noreply@blogger.com1tag:blogger.com,1999:blog-264654303037912405.post-42116045104284897512015-01-09T17:45:00.002+02:002015-01-09T17:46:33.339+02:00std::numeric_limits<UnsignedIntegral>::max() alternative<p>
<code>std::numeric_limits<uint32_t>::max()</code> == <code>static_cast<uint32_t>(-1)</code> == <code>~0U</code>
</p>Anonymoushttp://www.blogger.com/profile/14003906116514303118noreply@blogger.com4tag:blogger.com,1999:blog-264654303037912405.post-68259312333652635492014-12-08T14:44:00.000+02:002015-07-25T04:47:43.579+03:00How to detect VHD attached volume<p>
In order to detect a volume is VHD attached, you can query its device descriptor using <code>DeviceIoControl:</code>
</p>
<div dir="ltr" style="text-align: left;" trbidi="on">
<pre style="background: white; font-family: Consolas; font-size: 15px;"><span style="color: orangered;">#include</span> <span style="color: magenta;"><windows.h></span>
<span style="color: orangered;">#include</span> <span style="color: magenta;"><iostream></span>
<span style="color: navy;">using</span> <span style="color: navy;">namespace</span> <span style="color: #8080c0;">std</span><span style="color: #800040;">;</span>
<span style="color: navy;">int</span> <span style="color: #6f008a;">_tmain</span><span style="color: #800040;">(</span><span style="color: navy;">int</span> <span style="color: saddlebrown;">argc</span><span style="color: #800040;">,</span> <span style="color: #892be0;">_TCHAR</span><span style="color: #800040;">*</span> <span style="color: saddlebrown;">argv</span><span style="color: #800040;">[])</span>
<span style="color: #800040;">{</span>
<span style="color: navy;">if</span> <span style="color: #800040;">(</span><span style="color: saddlebrown;">argc</span> <span style="color: #800040;"><</span> <span style="color: darkgreen;">2</span><span style="color: #800040;">)</span> <span style="color: #800040;">{</span>
<span style="color: #804000;">cerr</span> <span style="color: #800040;"><<</span> <span style="color: magenta;">"Usage: FsUtil.exe file"</span> <span style="color: #800040;"><<</span> <span style="color: crimson;">endl</span><span style="color: #800040;">;</span>
<span style="color: navy;">return</span> <span style="color: #800040;">-</span><span style="color: darkgreen;">1</span><span style="color: #800040;">;</span>
<span style="color: #800040;">}</span>
<span style="color: navy;">auto</span> <span style="color: navy;">const</span> <span style="color: lightslategrey;">h</span> <span style="color: #800040;">=</span> <span style="color: #800040;">::</span><span style="color: #6f008a;">CreateFile</span><span style="color: #800040;">(</span><span style="color: saddlebrown;">argv</span><span style="color: #800040;">[</span><span style="color: darkgreen;">1</span><span style="color: #800040;">],</span> <span style="color: #6f008a;">GENERIC_READ</span><span style="color: #800040;">,</span> <span style="color: #6f008a;">FILE_SHARE_READ</span><span style="color: #800040;">,</span> <span style="color: navy;">nullptr</span><span style="color: #800040;">,</span> <span style="color: #6f008a;">OPEN_EXISTING</span><span style="color: #800040;">,</span> <span style="color: darkgreen;">0</span><span style="color: #800040;">,</span> <span style="color: navy;">nullptr</span><span style="color: #800040;">);</span>
<span style="color: navy;">if</span> <span style="color: #800040;">(</span><span style="color: lightslategrey;">h</span> <span style="color: #800040;">==</span> <span style="color: #6f008a;">INVALID_HANDLE_VALUE</span><span style="color: #800040;">)</span> <span style="color: #800040;">{</span>
<span style="color: #804000;">cerr</span> <span style="color: #800040;"><<</span> <span style="color: magenta;">"Unable to open "</span> <span style="color: #800040;"><<</span> <span style="color: saddlebrown;">argv</span><span style="color: #800040;">[</span><span style="color: darkgreen;">1</span><span style="color: #800040;">]</span> <span style="color: #800040;"><<</span> <span style="color: crimson;">endl</span><span style="color: #800040;">;</span>
<span style="color: navy;">return</span> <span style="color: #800040;">-</span><span style="color: darkgreen;">2</span><span style="color: #800040;">;</span>
<span style="color: #800040;">}</span>
<span style="color: #892be0;">STORAGE_PROPERTY_QUERY</span> <span style="color: lightslategrey;">spq</span> <span style="color: #800040;">=</span> <span style="color: #800040;">{</span> <span style="color: olive;">StorageDeviceProperty</span><span style="color: #800040;">,</span> <span style="color: olive;">PropertyStandardQuery</span> <span style="color: #800040;">};</span>
<span style="color: navy;">unsigned</span> <span style="color: navy;">char</span> <span style="color: lightslategrey;">b</span><span style="color: #800040;">[</span><span style="color: darkgreen;">1024</span><span style="color: #800040;">];</span>
<span style="color: navy;">auto</span> <span style="color: navy;">const</span> <span style="color: lightslategrey;">btyped</span> <span style="color: #800040;">=</span> <span style="color: navy;">reinterpret_cast</span><span style="color: #800040;"><</span><span style="color: #892be0;">STORAGE_DEVICE_DESCRIPTOR</span> <span style="color: #800040;">*>(&</span><span style="color: lightslategrey;">b</span><span style="color: #800040;">[</span><span style="color: darkgreen;">0</span><span style="color: #800040;">]);</span>
<span style="color: #892be0;">DWORD</span> <span style="color: lightslategrey;">br</span><span style="color: #800040;">;</span>
<span style="color: navy;">if</span> <span style="color: #800040;">(::</span><span style="color: crimson;">DeviceIoControl</span><span style="color: #800040;">(</span><span style="color: lightslategrey;">h</span><span style="color: #800040;">,</span> <span style="color: #6f008a;">IOCTL_STORAGE_QUERY_PROPERTY</span><span style="color: #800040;">,</span> <span style="color: #800040;">&</span><span style="color: lightslategrey;">spq</span><span style="color: #800040;">,</span> <span style="color: navy;">sizeof</span><span style="color: #800040;">(</span><span style="color: lightslategrey;">spq</span><span style="color: #800040;">),</span> <span style="color: #800040;">&</span><span style="color: lightslategrey;">b</span><span style="color: #800040;">[</span><span style="color: darkgreen;">0</span><span style="color: #800040;">],</span> <span style="color: navy;">sizeof</span><span style="color: #800040;">(</span><span style="color: lightslategrey;">b</span><span style="color: #800040;">),</span> <span style="color: #800040;">&</span><span style="color: lightslategrey;">br</span><span style="color: #800040;">,</span> <span style="color: navy;">nullptr</span><span style="color: #800040;">)</span> <span style="color: #800040;">==</span> <span style="color: #6f008a;">FALSE</span><span style="color: #800040;">)</span> <span style="color: #800040;">{</span>
<span style="color: #804000;">cerr</span> <span style="color: #800040;"><<</span> <span style="color: magenta;">"DeviceIoControl failed with "</span> <span style="color: #800040;"><<</span> <span style="color: #800040;">::</span><span style="color: crimson;">GetLastError</span><span style="color: #800040;">()</span> <span style="color: #800040;"><<</span> <span style="color: crimson;">endl</span><span style="color: #800040;">;</span>
<span style="color: navy;">return</span> <span style="color: #800040;">-</span><span style="color: darkgreen;">3</span><span style="color: #800040;">;</span>
<span style="color: #800040;">}</span>
<span style="color: navy;">auto</span> <span style="color: navy;">const</span> <span style="color: lightslategrey;">vendor</span> <span style="color: #800040;">=</span> <span style="color: navy;">reinterpret_cast</span><span style="color: #800040;"><</span><span style="color: #892be0;">LPCSTR</span><span style="color: #800040;">>(</span><span style="color: lightslategrey;">b</span> <span style="color: #800040;">+</span> <span style="color: lightslategrey;">btyped</span><span style="color: #800040;">-></span><span style="color: #20b08a;">VendorIdOffset</span><span style="color: #800040;">);</span>
<span style="color: navy;">auto</span> <span style="color: navy;">const</span> <span style="color: lightslategrey;">product</span> <span style="color: #800040;">=</span> <span style="color: navy;">reinterpret_cast</span><span style="color: #800040;"><</span><span style="color: #892be0;">LPCSTR</span><span style="color: #800040;">>(</span><span style="color: lightslategrey;">b</span> <span style="color: #800040;">+</span> <span style="color: lightslategrey;">btyped</span><span style="color: #800040;">-></span><span style="color: #20b08a;">ProductIdOffset</span><span style="color: #800040;">);</span>
<span style="color: #804000;">cout</span> <span style="color: #800040;"><<</span> <span style="color: magenta;">"Bus type: "</span> <span style="color: #800040;"><<</span> <span style="color: lightslategrey;">btyped</span><span style="color: #800040;">-></span><span style="color: #20b08a;">BusType</span> <span style="color: #800040;"><<</span> <span style="color: crimson;">endl</span>
<span style="color: #800040;"><<</span> <span style="color: magenta;">"Vendor: "</span> <span style="color: #800040;"><<</span> <span style="color: lightslategrey;">vendor</span> <span style="color: #800040;"><<</span> <span style="color: crimson;">endl</span>
<span style="color: #800040;"><<</span> <span style="color: magenta;">"Product: "</span> <span style="color: #800040;"><<</span> <span style="color: lightslategrey;">product</span> <span style="color: #800040;"><<</span> <span style="color: crimson;">endl</span><span style="color: #800040;">;</span>
<span style="color: navy;">return</span> <span style="color: darkgreen;">0</span><span style="color: #800040;">;</span>
<span style="color: #800040;">}</span></pre>
</div>
<p>
<code>BusType</code> would be <code>BusTypeFileBackedVirtual</code>, vendor - Msft, product - Virtual Disk.
</p>Anonymoushttp://www.blogger.com/profile/14003906116514303118noreply@blogger.com2tag:blogger.com,1999:blog-264654303037912405.post-24442299747702219232014-10-03T15:52:00.000+03:002014-10-03T15:52:03.751+03:00Visual C++ versus new T[N] { 0 }<pre>
auto const p = std::unique_ptr<unsigned char[]>(new unsigned char[1024 * 50] { 0 });
</pre>
<p>
<code>link.exe</code> зависает, кушая при этом процессор.
</p>
<pre>
auto const p = std::unique_ptr<unsigned char[]>(new unsigned char[1024 * 50] { 0 });
</pre>
<p>
Результат: <code>fatal error C1063: compiler limit : compiler stack overflow</code>
</p>
<pre>
auto const p = std::unique_ptr<unsigned char[]>(new unsigned char[1024] { 0 });
</pre>
<p>
Сгенерированные инструкции:
</p>
<pre>
00007FF6D1CF1A3A call operator new[] (07FF6D1CF1E30h)
00007FF6D1CF1A3F xor ecx,ecx
00007FF6D1CF1A41 test rax,rax
00007FF6D1CF1A44 je wmain+383h (07FF6D1CF1D9Bh)
00007FF6D1CF1A4A mov qword ptr [rax],rcx
00007FF6D1CF1A4D mov qword ptr [rax+8],rcx
00007FF6D1CF1A51 mov qword ptr [rax+10h],rcx
00007FF6D1CF1A55 mov qword ptr [rax+18h],rcx
00007FF6D1CF1A59 mov qword ptr [rax+20h],rcx
00007FF6D1CF1A5D mov qword ptr [rax+28h],rcx
00007FF6D1CF1A61 mov qword ptr [rax+30h],rcx
00007FF6D1CF1A65 mov qword ptr [rax+38h],rcx
00007FF6D1CF1A69 mov qword ptr [rax+40h],rcx
00007FF6D1CF1A6D mov qword ptr [rax+48h],rcx
00007FF6D1CF1A71 mov qword ptr [rax+50h],rcx
00007FF6D1CF1A75 mov qword ptr [rax+58h],rcx
00007FF6D1CF1A79 mov qword ptr [rax+60h],rcx
00007FF6D1CF1A7D mov qword ptr [rax+68h],rcx
00007FF6D1CF1A81 mov qword ptr [rax+70h],rcx
00007FF6D1CF1A85 mov qword ptr [rax+78h],rcx
00007FF6D1CF1A89 mov qword ptr [rax+80h],rcx
...
</pre>
<p>
Заполнение пасяти нулями шмомпилятор от Быдлософт развернул в 128 инструкций <code>mov</code>.
</p>
<p>
То же, но компилятором Intel C++:
</p>
<pre>
00007FF755B1102B call operator new[] (07FF755B13C50h)
...
00007FF755B11053 call _intel_fast_memcpy (07FF755B11B50h)
</pre>
<p>
<code>_intel_fast_memcpy</code> для моего Core i5 2XXX выбрала реализацию с циклом следующего вида:
</p>
<pre>
00007FF755B132A0 movdqa xmm0,xmmword ptr [rdx]
00007FF755B132A4 movdqa xmm1,xmmword ptr [rdx+10h]
00007FF755B132A9 movdqa xmmword ptr [rcx],xmm0
00007FF755B132AD movdqa xmmword ptr [rcx+10h],xmm1
00007FF755B132B2 lea r8,[r8-80h]
00007FF755B132B6 movdqa xmm2,xmmword ptr [rdx+20h]
00007FF755B132BB movdqa xmm3,xmmword ptr [rdx+30h]
00007FF755B132C0 movdqa xmmword ptr [rcx+20h],xmm2
00007FF755B132C5 movdqa xmmword ptr [rcx+30h],xmm3
00007FF755B132CA movdqa xmm0,xmmword ptr [rdx+40h]
00007FF755B132CF movdqa xmm1,xmmword ptr [rdx+50h]
00007FF755B132D4 cmp r8,0A8h
00007FF755B132DB movdqa xmmword ptr [rcx+40h],xmm0
00007FF755B132E0 movdqa xmmword ptr [rcx+50h],xmm1
00007FF755B132E5 movdqa xmm2,xmmword ptr [rdx+60h]
00007FF755B132EA movdqa xmm3,xmmword ptr [rdx+70h]
00007FF755B132EF lea rdx,[rdx+80h]
00007FF755B132F6 movdqa xmmword ptr [rcx+60h],xmm2
00007FF755B132FB movdqa xmmword ptr [rcx+70h],xmm3
00007FF755B13300 lea rcx,[rcx+80h]
00007FF755B13307 jge __intel_memcpy+0E90h (07FF755B132A0h)
</pre>
<p>
А если выбрать Favor Small Code в настройках компилятора, то заполнение нулями превращается в ожидаемый и привычный <code>rep movs</code>:
</p>
<pre>
...
00007FF615041053 rep movs qword ptr [rdi],qword ptr [rsi]
</pre>Anonymoushttp://www.blogger.com/profile/14003906116514303118noreply@blogger.com0tag:blogger.com,1999:blog-264654303037912405.post-54306328311718846042014-06-16T19:35:00.001+03:002014-06-16T19:36:55.720+03:00Nosgoth beta keys<p>
VEBOME-957-JAMONO-230
</p>
<p>
NABOFA-205-CAXABU-021
</p>
<p>
VEJODE-375-COZABO-211
</p>Anonymoushttp://www.blogger.com/profile/14003906116514303118noreply@blogger.com0tag:blogger.com,1999:blog-264654303037912405.post-11191566127436277672014-06-12T16:30:00.001+03:002014-06-12T16:30:22.433+03:00WinDbg: find probable CONTEXT records<p>This script finds and pretty prints all probable <a href="http://msdn.microsoft.com/en-us/library/windows/desktop/ms679284(v=vs.85).aspx" target="_blank"><code>CONTEXT</code></a> struct instances throughout x64 process address space:</p>
<pre>
0:000> .foreach ( CxrPtr { s -[w1]b 0x00000000000000000 L?FFFFFFFFFFFFFFFF 2b 00 2b 00 53 00 2b 00 } ) { .cxr ${CxrPtr}-@@(#FIELD_OFFSET(ntdll!_CONTEXT, SegDs)) }
rax=000000000f2907e0 rbx=00000001420b70f0 rcx=0000000010c3d130
rdx=0000000010c3cad8 rsi=00000001420b7d08 rdi=000000013fda9cb0
rip=000007fe99e71cc9 rsp=0000000010c3e850 rbp=0000000010c3e870
r8=0000000010c2a000 r9=000000000f2907e0 r10=000007fef6bd6738
r11=0000000000000001 r12=0000000140e4fb00 r13=000000033fcc69f8
r14=0000000010c3f098 r15=0000000000000004
iopl=0 nv up ei pl nz na pe nc
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202
000007fe`99e71cc9 8a4510 mov al,byte ptr [rbp+10h] ss:00000000`10c3e880=00
...
rax=000000004685a478 rbx=0000000241d02878 rcx=0000000000000000
rdx=0000000000000000 rsi=0000000241c9d708 rdi=0000000241d02838
rip=000007fe9d9d39f4 rsp=000000004685a450 rbp=000000004685a4a0
r8=0000000441b49850 r9=0000000000000000 r10=000007fe9b1e1ac0
r11=0000000441b49870 r12=0000000241c90e88 r13=000007fe9b299448
r14=00000001406cc858 r15=0000000441b24af0
iopl=0 nv up ei pl nz na po nc
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010206
000007fe`9d9d39f4 803900 cmp byte ptr [rcx],0 ds:00000000`00000000=??
</pre>
<p>
Then you would normally use RIP and RSP registers to find relevant code and thread context:
</p>
<pre>
0:000> !IP2MD 000007fe99e71cc9
MethodDesc: 000007fe98e229c0
Method Name: Replay.Core.Implementation.AutomaticUpdate.PatchDetector.IsPatched(System.Diagnostics.FileVersionInfo)
Class: 000007fe98dec4a0
MethodTable: 000007fe98e22a70
mdToken: 0000000006000463
Module: 000007fe988acb20
IsJitted: yes
CodeAddr: 000007fe99e71c50
Transparency: Critical
0:000> !IP2MD 000007fefde1940d
Failed to request MethodData, not in JIT code range
0:000> ln 000007fefde1940d
(000007fe`fde193d0) KERNELBASE!RaiseException+0x39 | (000007fe`fde19420) KERNELBASE!CreateMutexExW
0:000> !address 000000004685a450
Mapping file section regions...
Mapping module regions...
Mapping PEB regions...
Mapping TEB and stack regions...
Mapping heap regions...
Mapping page heap regions...
Mapping other regions...
Mapping stack trace database regions...
Mapping activation context regions...
Usage: Stack
Base Address: 00000000`46852000
End Address: 00000000`46860000
Region Size: 00000000`0000e000
State: 00001000 MEM_COMMIT
Protect: 00000004 PAGE_READWRITE
Type: 00020000 MEM_PRIVATE
Allocation Base: 00000000`46460000
Allocation Protect: 00000004 PAGE_READWRITE
More info: ~88k
0:000> ~88k
Child-SP RetAddr Call Site
00000000`4685ed48 000007fe`fde110dc ntdll!NtWaitForSingleObject+0xa
00000000`4685ed50 000007fe`f7e89622 KERNELBASE!WaitForSingleObjectEx+0x79
00000000`4685edf0 000007fe`f7e89841 clr!CLRSemaphore::Wait+0x8a
00000000`4685eeb0 000007fe`f7e897ec clr!ThreadpoolMgr::UnfairSemaphore::Wait+0x134
00000000`4685eef0 000007fe`f7d733de clr!ThreadpoolMgr::WorkerThreadStart+0x204
00000000`4685efb0 00000000`77a959ed clr!Thread::intermediateThreadProc+0x7d
00000000`4685fb70 00000000`77ccc541 kernel32!BaseThreadInitThunk+0xd
00000000`4685fba0 00000000`00000000 ntdll!RtlUserThreadStart+0x1d
</pre>Anonymoushttp://www.blogger.com/profile/14003906116514303118noreply@blogger.com1tag:blogger.com,1999:blog-264654303037912405.post-934269646516290722014-06-12T16:21:00.002+03:002014-06-12T16:21:33.380+03:00Apple Swift<a href="https://fbcdn-sphotos-d-a.akamaihd.net/hphotos-ak-xfp1/t1.0-9/10371755_10152405897019651_1013040155042295396_n.jpg" imageanchor="1" ><img border="0" src="https://fbcdn-sphotos-d-a.akamaihd.net/hphotos-ak-xfp1/t1.0-9/10371755_10152405897019651_1013040155042295396_n.jpg" /></a>Anonymoushttp://www.blogger.com/profile/14003906116514303118noreply@blogger.com1tag:blogger.com,1999:blog-264654303037912405.post-30614673414759791202014-05-27T18:18:00.000+03:002014-05-27T18:18:04.342+03:00Windows Internals 7th edition<p>
<a href="http://www.amazon.com/Windows-Internals-Book-User-Edition/dp/0735684189"><strong>March 13, 2015</strong></a>
</p>Anonymoushttp://www.blogger.com/profile/14003906116514303118noreply@blogger.com0tag:blogger.com,1999:blog-264654303037912405.post-52782641775778584202014-01-16T01:02:00.003+02:002014-01-16T01:02:29.669+02:00KiPageFault into BSOD when stepping over <p>
I've been struggling long time with weird bug check during kernel driver debugging. Stack trace would look like this:
</p>
<pre class="listing">
1: kd> k
Child-SP RetAddr Call Site
ffffd000`20463d78 fffff800`1aa610ea nt!DbgBreakPointWithStatus
ffffd000`20463d80 fffff800`1aa609fb nt!KiBugCheckDebugBreak+0x12
ffffd000`20463de0 fffff800`1a9d8da4 nt!KeBugCheck2+0x8ab
ffffd000`204644f0 fffff800`1aa00b1f nt!KeBugCheckEx+0x104
ffffd000`20464530 fffff800`1a8c75ad nt! ?? ::FNODOBFM::`string'+0x1797f
ffffd000`204645d0 fffff800`1a9e2f2f nt!MmAccessFault+0x7ed
ffffd000`20464710 fffff800`002b92e3 nt!KiPageFault+0x12f
ffffd000`204648a0 fffff800`0117b41f Wdf01000!imp_WdfFdoInitQueryProperty+0x28
ffffd000`204648f0 fffff800`0118117f MyVolFlt!WdfFdoInitQueryProperty+0x5f [c:\program files (x86)\windows kits\8.1\include\wdf\kmdf\1.13\wdffdo.h @ 217]
ffffd000`20464940 fffff800`0027f55b MyVolFlt!MyVolFltEvtDeviceAdd+0x9f [c:\development\projects\kernelmode\myvolflt\driver.c @ 116]
ffffd000`20464bd0 fffff800`1a9539d9 Wdf01000!FxDriver::AddDevice+0xab
ffffd000`20464ff0 fffff800`1ace18ab nt!PpvUtilCallAddDevice+0x35
ffffd000`20465030 fffff800`1acdff9e nt!PnpCallAddDevice+0x63
ffffd000`204650b0 fffff800`1acdf2db nt!PipCallDriverAddDevice+0x6e2
ffffd000`20465250 fffff800`1ad14b89 nt!PipProcessDevNodeTree+0x1cf
ffffd000`204654d0 fffff800`1a97d0b8 nt!PiProcessReenumeration+0x91
ffffd000`20465520 fffff800`1a97cf2e nt!PnpDeviceActionWorker+0x168
ffffd000`204655d0 fffff800`1af93382 nt!PnpRequestDeviceAction+0x1da
ffffd000`20465610 fffff800`1af89022 nt!IopInitializeBootDrivers+0x83e
ffffd000`204658b0 fffff800`1af7794d nt!IoInitSystem+0x91e
ffffd000`204659d0 fffff800`1ad7bd09 nt!Phase1InitializationDiscard+0xe61
ffffd000`20465bd0 fffff800`1a9182e4 nt!Phase1Initialization+0x9
ffffd000`20465c00 fffff800`1a9df2c6 nt!PspSystemThreadStartup+0x58
ffffd000`20465c60 00000000`00000000 nt!KiStartSystemThread+0x16
</pre>
<p>Bug Check description:</p>
<pre class="listing">
1: kd> !analyze -v
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************
PAGE_FAULT_IN_NONPAGED_AREA (50)
Invalid system memory was referenced. This cannot be protected by try-except,
it must be protected by a Probe. Typically the address is just plain bad or it
is pointing at freed memory.
Arguments:
Arg1: ffffe00020464c10, memory referenced.
...
</pre>
<p>Now lets see what is this address:</p>
<pre class="listing">
1: kd> !pool ffffe00020464c10
Pool page ffffe00020464c10 region is Nonpaged pool
ffffe00020464000 is not a valid large pool allocation, checking large session pool...
Unable to read large session pool table (Session data is not present in mini and kernel-only dumps)
ffffe00020464000 is not valid pool. Checking for freed (or corrupt) pool
Address ffffe00020464000 could not be read. It may be a freed, invalid or paged out page
1: kd> ? poi(DeviceInit)
Evaluate expression: -35183830610928 = ffffe000`20464c10
</pre>
<p>
Wow, faulting memory references is <code>DeviceInit</code> actually! And it is located on stack (because of KMDF model).
</p>
<p>Sure IRQL is at PASSIVE level:</p>
<pre class="listing">
1: kd> !irql
Debugger saved IRQL for processor 0x1 -- 0 (LOW_LEVEL)
</pre>
<p>The funniest thing so far is that if I set bp after the call to <code>WdfFdoInitQueryProperty</code> - it would run smoothly. So there is something wrong with the debugger interacting OS kernel.</p>
<p>Now I finally managed to figure out what was wrong. I would normally set my bp during initial break-in sequence:</p>
<pre class="listing">
Connected to Windows 8 9600 x64 target at (Thu Jan 16 00:54:33.435 2014 (UTC + 2:00)), ptr64 TRUE
Kernel Debugger connection established.
************* Symbol Path validation summary **************
Response Time (ms) Location
Deferred cache*C:\Development\Tools\Symbols
Deferred srv*http://msdl.microsoft.com/download/symbols
Symbol search path is: cache*C:\Development\Tools\Symbols;srv*http://msdl.microsoft.com/download/symbols
Executable search path is:
Windows 8 Kernel Version 9600 MP (1 procs) Free x64
Built by: 9600.16452.amd64fre.winblue_gdr.131030-1505
Machine Name:
Kernel base = 0xfffff800`5547e000 PsLoadedModuleList = 0xfffff800`55742990
System Uptime: 0 days 0:00:00.102
nt!DebugService2+0x5:
fffff800`555d28e5 cc int 3
kd> bp MyVolFltEvtDeviceAdd
kd> g
</pre>
<p>And here what happens after:</p>
<pre class="listing">
Unload module \SystemRoot\system32\mcupdate_GenuineIntel.dll at fffff800`1b200000
Unload module \SystemRoot\System32\drivers\werkernel.sys at fffff800`19ed5000
...
Unload module \SystemRoot\system32\DRIVERS\MyVolFlt.sys at fffff800`1b9ed000
nt!DebugService2+0x5:
fffff800`555d28e5 cc int 3
kd> k
# Child-SP RetAddr Call Site
00 fffff800`573991a8 fffff800`55544361 nt!DebugService2+0x5
01 fffff800`573991b0 fffff800`555442ff nt!DbgLoadImageSymbols+0x45
02 fffff800`57399200 fffff800`55b76fc4 nt!DbgLoadImageSymbolsUnicode+0x2b
03 fffff800`57399240 fffff800`55b7684b nt!MiReloadBootLoadedDrivers+0x300
04 fffff800`573993c0 fffff800`55b6c091 nt!MiInitializeDriverImages+0x163
05 fffff800`57399470 fffff800`55b67299 nt!MiInitSystem+0x3d9
06 fffff800`57399500 fffff800`557e84ea nt!InitBootProcessor+0x301
07 fffff800`57399740 fffff800`557de1a3 nt!KiInitializeKernel+0x5a2
08 fffff800`57399ad0 00000000`00000000 nt!KiSystemStartup+0x193
</pre>
<p>It is unloading boot time drivers! And reloading with different start addresses! So when I set my breakpoint at <code>MyVolFltEvtDeviceAdd</code>, WinDbg would insert <code>int 3</code> instruction and during module relocation that instruction is copied as is. So my breakpoint actually hits, despite code relocation. But this is where the Windows and debugger fall apart - they don't know about this breakpoint.</p>
<p>In order to issue correct breakpoint address, you must break on module load:</p>
<pre class="listing">
kd> sxe ld MyVolFlt
kd> sxe ud MyVolFlt
kd> sx
ct - Create thread - ignore
et - Exit thread - ignore
cpr - Create process - ignore
epr - Exit process - ignore
ld - Load module - break
(only break for myvolflt)
ud - Unload module - break
(only break for MyVolFlt)
</pre>
<p>And issue <code>bp</code> command after kernel reloads boot loaded drivers.</p>Anonymoushttp://www.blogger.com/profile/14003906116514303118noreply@blogger.com0tag:blogger.com,1999:blog-264654303037912405.post-5984917651405589642014-01-10T20:05:00.001+02:002014-01-10T21:07:33.220+02:00Metal Gear Rising: Revengeance<p>Metal Gear Rising: Revengeance has just released on PC!</p>
<p>Amazing OST:</p>
<div dir="ltr" style="text-align: left;" trbidi="on">
<object width="320" height="266" class="BLOGGER-youtube-video" classid="clsid:D27CDB6E-AE6D-11cf-96B8-444553540000" codebase="http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=6,0,40,0" data-thumbnail-src="https://ytimg.googleusercontent.com/vi/0vp_4vc-btI/0.jpg"><param name="movie" value="https://youtube.googleapis.com/v/0vp_4vc-btI&source=uds" /><param name="bgcolor" value="#FFFFFF" /><param name="allowFullScreen" value="true" /><embed width="320" height="266" src="https://youtube.googleapis.com/v/0vp_4vc-btI&source=uds" type="application/x-shockwave-flash" allowfullscreen="true"></embed></object></div>
Anonymoushttp://www.blogger.com/profile/14003906116514303118noreply@blogger.com0tag:blogger.com,1999:blog-264654303037912405.post-43252454291220878642013-12-11T15:38:00.002+02:002013-12-17T21:48:15.289+02:00ReaderWriterLockSlim fails on dual-socket environments <p>This is yet another story of orphaned <code>ReaderWriterLockSlim</code>.</p>
<p>Another dump, <a href="http://chabster.blogspot.com/2013/07/a-story-of-orphaned-readerwriterlockslim.html" title="A story of orphaned ReaderWriterLockSlim" target="_blank">the same problem</a> - <code>ReaderWriterLockSlim</code> object state is corrupted:</p>
<pre>
0:173> !do 0x0000000001c679f8
Name: System.Threading.ReaderWriterLockSlim
MethodTable: 000007f87ec7c1d8
EEClass: 000007f87e999448
Size: 96(0x60) bytes
File: C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Core\v4.0_4.0.0.0__b77a5c561934e089\System.Core.dll
Fields:
MT Field Offset Type VT Attr Value Name
000007f880dbcbf0 4000755 50 System.Boolean 1 instance 1 fIsReentrant
000007f880dbe0c8 4000756 30 System.Int32 1 instance 0 myLock
000007f880db2308 4000757 34 System.UInt32 1 instance 1 numWriteWaiters
000007f880db2308 4000758 38 System.UInt32 1 instance 28 numReadWaiters
000007f880db2308 4000759 3c System.UInt32 1 instance 0 numWriteUpgradeWaiters
000007f880db2308 400075a 40 System.UInt32 1 instance 0 numUpgradeWaiters
000007f880dbcbf0 400075b 51 System.Boolean 1 instance 0 fNoWaiters
000007f880dbe0c8 400075c 44 System.Int32 1 instance -1 upgradeLockOwnerId
000007f880dbe0c8 400075d 48 System.Int32 1 instance -1 writeLockOwnerId
000007f880db9138 400075e 8 ...g.EventWaitHandle 0 instance 000000000381eb38 writeEvent
000007f880db9138 400075f 10 ...g.EventWaitHandle 0 instance 00000000035a32e0 readEvent
000007f880db9138 4000760 18 ...g.EventWaitHandle 0 instance 0000000000000000 upgradeEvent
000007f880db9138 4000761 20 ...g.EventWaitHandle 0 instance 0000000000000000 waitUpgradeEvent
000007f880dd0398 4000763 28 System.Int64 1 instance 9 lockID
000007f880dbcbf0 4000765 52 System.Boolean 1 instance 0 fUpgradeThreadHoldingRead
000007f880db2308 4000766 4c System.UInt32 1 instance <strong>1073741824</strong> owners
000007f880dbcbf0 4000767 53 System.Boolean 1 instance 0 fDisposed
000007f880dd0398 4000762 408 System.Int64 1 static 14118 s_nextLockID
000007f87ec99a20 4000764 8 ...ReaderWriterCount 0 TLstatic t_rwc
0:173> .formats 0n1073741824
Evaluate expression:
Hex: 00000000`40000000
</pre>
<p><code>EnterReadLock</code>, <code>EnterWriteLock</code> and other <code>Enter</code> operations waiting for an event which never goes off. Deadlock.</p>
<p>I must say that I checked possibilities of thread aborts in this code and found no signs of such scenarios happening. This made me desperately searching for another root cause of the problem.</p>
<p>So started searching <code>ReaderWriterLockSlim.cs</code> file for potential problems. I immediately became suspicious when I realyzed there is lack of synchronization when, for example, <code>TryEnterUpgradeableReadLockCore</code> method modified one of object fields:</p>
<pre>
uint owners;
...
private bool TryEnterReadLockCore(TimeoutTracker timeout)
{
...
owners++;
}
</pre>
<p>Fields are not declared volatile, nor are they modified via interlocked operations. The only exception is the <code>myLock</code> field, which is used as a spin lock ang modified via <code>Interlocked.CompareExchange</code>:</p>
<pre>
[MethodImpl(MethodImplOptions.AggressiveInlining)]
private void EnterMyLock()
{
if (Interlocked.CompareExchange(ref myLock, 1, 0) != 0)
EnterMyLockSpin();
}
</pre>
<p>Note, however, spin lock release method doesn't use <code>Interlocked</code> operation:</p>
<pre>
private void ExitMyLock()
{
Debug.Assert(myLock != 0, "Exiting spin lock that is not held");
myLock = 0;
}
</pre>
<p>
This looks to be a mistake, possibly the root cause on one of root causes.
</p>
<p>OK, lets go back to the problem - <code>ReaderWriterLockSlim</code> gets locked forever on 24-core dual socket Intel hardware. Threads are not aborted, the code is perfect. So what the hell is going on?</p>
<p>Well, the problem looks to be bad software (<code>ReaderWriterLockSlim</code>) on expensive hardware. Dell PowerEdge R720 has two psysical CPUs - 2x Intel Xeon E5-2620, 1200 MHz (12 x 100), 6 cores and 12 threads each. 24 logical cores total. And the problem is experienced only on such configurations.</p>
<p>I made a program that creates 24 (= <code>Environment.ProcessorCount</code>) threads with highest priority acquiring and releasing the lock in a tight loop:</p>
<pre>
using System;
using System.Collections.Generic;
using System.Runtime.CompilerServices;
using System.Threading;
namespace RWLSTest
{
internal class Program
{
private static readonly ReaderWriterLockSlim slim = new ReaderWriterLockSlim(LockRecursionPolicy.SupportsRecursion);
private static readonly List<object> objects = new List<object>();
private static readonly Int32 processorCount = Environment.ProcessorCount;
private static Int32 threadsCount;
private static Int64 reads;
private static Int64 writes;
private static volatile Object[] threads = new Object[processorCount];
private static Action loopAction;
static Program()
{
// Let it JIT those methods
using (var temp = new ReaderWriterLockSlim(LockRecursionPolicy.SupportsRecursion)) {
Thread.Yield();
temp.EnterReadLock();
temp.ExitReadLock();
}
var thread = new Thread(() =>
{
try {
Thread.Sleep(Timeout.Infinite);
}
catch {
return;
}
throw new InvalidOperationException();
});
thread.Start();
try {
thread.Abort();
}
catch (Exception e) {
Console.WriteLine(e.Message);
}
}
private static void LoopWithEmptryTryBlocks()
{
var random = new Random(Environment.TickCount);
for (;;) {
if (random.Next(processorCount) <= (processorCount / 4)) {
Interlocked.Increment(ref writes);
try {}
finally {
slim.EnterWriteLock();
}
try {
ExclusiveLoop(random);
}
finally {
slim.ExitWriteLock();
}
}
else {
Interlocked.Increment(ref reads);
try {}
finally {
slim.EnterReadLock();
}
try {
SharedLoop(random);
}
finally {
slim.ExitReadLock();
}
}
}
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
private static void SharedLoop(Random random)
{
foreach (var o in objects) {
var i = (Int32)o;
if ((i % processorCount) == (random.Next() % processorCount) && random.Next(37) == 3) {
break;
}
}
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
private static void ExclusiveLoop(Random random)
{
if (objects.Count < 10240) {
for (var i = 0; i < 19; ++i) {
if (random.Next(13) == 7) {
objects.Add(random.Next());
}
}
}
for (var i = 0; i < 13; ++i) {
if (objects.Count > 0 && random.Next(19) == 13) {
objects.Remove(random.Next() % objects.Count);
}
}
}
private static void Loop()
{
var random = new Random(Environment.TickCount);
for (;;) {
if (random.Next(processorCount) <= (processorCount / 4)) {
slim.EnterWriteLock();
try {
ExclusiveLoop(random);
}
finally {
slim.ExitWriteLock();
}
}
else {
slim.EnterReadLock();
try {
SharedLoop(random);
}
finally {
slim.ExitReadLock();
}
}
}
}
private static void StartOneThread(Object state)
{
var thread = new Thread(() =>
{
try {
Interlocked.Increment(ref threadsCount);
loopAction();
}
catch (ThreadAbortException) {}
finally {
Interlocked.Decrement(ref threadsCount);
ThreadPool.UnsafeQueueUserWorkItem(StartOneThread, state);
}
}) { Priority = ThreadPriority.Highest };
thread.Start();
Thread.VolatileWrite(ref threads[(Int32)state], thread);
}
private static void Main(string[] args)
{
var random = new Random(Environment.TickCount);
var abortCycle = 0;
if (args.Length > 0) {
abortCycle = Int32.Parse(args[0]);
loopAction = LoopWithEmptryTryBlocks;
}
else {
loopAction = Loop;
}
for (var i = 0; i < processorCount; ++i) {
StartOneThread(i);
}
for (var i = 0U;; ++i) {
Thread.Sleep(1);
if (abortCycle > 0 && i % abortCycle == 0) {
var ti = random.Next(111) % processorCount;
var thread = (Thread)Thread.VolatileRead(ref threads[ti]);
if (thread != null) {
Console.WriteLine("Aborting thread #" + ti);
try {
thread.Abort();
}
catch (Exception e) {
Console.WriteLine(e.Message);
}
}
}
}
}
}
}
</pre>
<p>I ran it several times and after about 1 hour all threads ended up waiting for lock event to fire. Voila! Have a look at the state of <code>ReaderWriterLockSlim</code> object:</p>
<pre>
0:000> !do 000000b343bd2860
Name: System.Threading.ReaderWriterLockSlim
MethodTable: 000007fbf887c1a8
EEClass: 000007fbf8599448
Size: 96(0x60) bytes
File: C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Core\v4.0_4.0.0.0__b77a5c561934e089\System.Core.dll
Fields:
MT Field Offset Type VT Attr Value Name
000007fbfe6ac7b8 4000755 50 System.Boolean 1 instance 1 fIsReentrant
000007fbfe6adc90 4000756 30 System.Int32 1 instance 0 myLock
000007fbfe6a1ed0 4000757 34 System.UInt32 1 instance 1 numWriteWaiters
000007fbfe6a1ed0 4000758 38 System.UInt32 1 instance 23 numReadWaiters
000007fbfe6a1ed0 4000759 3c System.UInt32 1 instance 0 numWriteUpgradeWaiters
000007fbfe6a1ed0 400075a 40 System.UInt32 1 instance 0 numUpgradeWaiters
000007fbfe6ac7b8 400075b 51 System.Boolean 1 instance 0 fNoWaiters
000007fbfe6adc90 400075c 44 System.Int32 1 instance -1 upgradeLockOwnerId
000007fbfe6adc90 400075d 48 System.Int32 1 instance -1 writeLockOwnerId
000007fbfe6a8d00 400075e 8 ...g.EventWaitHandle 0 instance 000000b343beb448 writeEvent
000007fbfe6a8d00 400075f 10 ...g.EventWaitHandle 0 instance 000000b343be2fd0 readEvent
000007fbfe6a8d00 4000760 18 ...g.EventWaitHandle 0 instance 0000000000000000 upgradeEvent
000007fbfe6a8d00 4000761 20 ...g.EventWaitHandle 0 instance 0000000000000000 waitUpgradeEvent
000007fbfe6bff60 4000763 28 System.Int64 1 instance 1 lockID
000007fbfe6ac7b8 4000765 52 System.Boolean 1 instance 0 fUpgradeThreadHoldingRead
000007fbfe6a1ed0 4000766 4c System.UInt32 1 instance <strong>1073741824</strong> owners
000007fbfe6ac7b8 4000767 53 System.Boolean 1 instance 0 fDisposed
000007fbfe6bff60 4000762 408 System.Int64 1 static 2 s_nextLockID
000007fbf88999f0 4000764 8 ...ReaderWriterCount 0 TLstatic t_rwc
</pre>
<p>There are 23 reader waiters, 1 writer waiter and owners field is <code>0x40000000</code> once again. All of 24 threads look like the following:</p>
<pre>
0:000> ~22e !CLRStack
OS Thread Id: 0xf28 (22)
Child SP IP Call Site
000000b361a1df78 000007fc137b315b [HelperMethodFrame_1OBJ: 000000b361a1df78] System.Threading.WaitHandle.WaitOneNative(System.Runtime.InteropServices.SafeHandle, UInt32, Boolean, Boolean)
000000b361a1e0a0 000007fbfe5195c4 System.Threading.WaitHandle.InternalWaitOne(System.Runtime.InteropServices.SafeHandle, Int64, Boolean, Boolean)
000000b361a1e0e0 000007fbf8af4c25 System.Threading.ReaderWriterLockSlim.WaitOnEvent(System.Threading.EventWaitHandle, UInt32 ByRef, TimeoutTracker)
000000b361a1e150 000007fbf8dd4c48 System.Threading.ReaderWriterLockSlim.TryEnterReadLockCore(TimeoutTracker)
000000b361a1e1b0 000007fbf8804d4a System.Threading.ReaderWriterLockSlim.TryEnterReadLock(TimeoutTracker)
000000b361a1e200 000007fbf8af55ad System.Threading.ReaderWriterLockSlim.TryEnterReadLock(Int32)
000000b361a1e250 000007fba0010a45 RWLSTest.Program.Loop()
000000b361a1e2c0 000007fba00106f7 RWLSTest.Program+<>c__DisplayClass4.<startonethread>b__3()
000000b361a1e330 000007fbfe4ff8a5 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
000000b361a1e490 000007fbfe4ff609 System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
000000b361a1e4c0 000007fbfe4ff5c7 System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
000000b361a1e510 000007fbfe512d21 System.Threading.ThreadHelper.ThreadStart()
000000b361a1e828 000007fbff6bf713 [GCFrame: 000000b361a1e828]
000000b361a1eb58 000007fbff6bf713 [DebuggerU2MCatchHandlerFrame: 000000b361a1eb58]
</pre>
<p>They all <code>WaitOnEvent</code>, but who and when will fire the event? This never happens. Deadlock.</p>
<p>
Now lets get back to <code>ExitMyLock</code>.
<code>ReaderWriterLockSlim</code> contains many fields with at least 88 bytes storage space required (without extra aligning, if needed). Modern Intel CPUs have cache lines of 64 bytes which is too small to entirely hold <code>ReaderWriterLockSlim</code> object instance.
So each one requires at least two cache lines to hold its data. Since the distance between <code>myLock</code> and <code>owners</code> fields is more than 64 bytes (both x86 and x64), releasing <code>myLock</code> without a memory barrier (or interlocked instruction) causes only a portion of object's storage invalidated on demand between CPU cores and/or CPUs.
Invalidation is forced by <code>EnterMyLock</code>'s interlocked instruction. But only 64 bytes of aligned memory where <code>myLock</code> resides. Other cache line's changes might not be visible at that point.
So the core acquiring the lock may see inconsistent object state.
</p>
<p>
Very important note: <code>ReaderWriterLockSlim.cs</code> is a part of <a href="http://referencesource.microsoft.com/netframework.aspx">4.5Update1</a> reference source. Vanilla .NET 4.5 and probably several updates following it has this code, for example <code>4.0.30319.17929</code>, <code>4.0.30319.18408</code>.
Recent versions, for example <code>4.0.30319.33440</code>, has fixed this:
</p>
<pre>
private void ExitMyLock()
{
Volatile.Write(ref myLock, 0);
}
</pre>
<p>Volatile write inserts explicit memory barriers and makes any changes visible to other cores and CPUs.</p>
<p class="warning"><strong>Conclusion: do not use <code>ReaderWriterLockSlim</code> class without .NET Framework updated to at least <code>4.0.30319.33440</code>. Its will eventually fail, at least on dual-socket Intel system.</strong></p>
<p class="warning"><strong>Windows 8.1 and Windows Server 2012 R2 have this issue fixed. Windows Server 2012 (nor R2) seems to stuck with buggy implementation of <code>ReaderWriterLockSlim</code> class. After installing all available updates, <code>ExitMyLock</code> looks the same (no volatile write operation).</strong></p>Anonymoushttp://www.blogger.com/profile/14003906116514303118noreply@blogger.com2tag:blogger.com,1999:blog-264654303037912405.post-49344997845463812512013-11-19T12:35:00.000+02:002013-11-19T12:35:19.358+02:00Injustice: Gods Among Us<p>
<a href="http://geekcity.ru/wp-content/uploads/2013/08/Injustice-Gods-Among-Us-Characters-Art.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="http://geekcity.ru/wp-content/uploads/2013/08/Injustice-Gods-Among-Us-Characters-Art.jpg" /></a>
<a href="http://geekcity.ru/injustice-gods-among-us-gajd-po-igre-onlajn/">
<q>Игроки в онлайне делятся на два типа: одни спамят, а другие занимались сексом с их матерью</q>
</a>
</p>Anonymoushttp://www.blogger.com/profile/14003906116514303118noreply@blogger.com0tag:blogger.com,1999:blog-264654303037912405.post-64402325052676449352013-11-15T14:41:00.000+02:002013-11-15T14:41:02.457+02:00Visual Studio 2012 Update 4
Anonymoushttp://www.blogger.com/profile/14003906116514303118noreply@blogger.com1tag:blogger.com,1999:blog-264654303037912405.post-37989964192385561202013-11-15T10:09:00.000+02:002013-11-15T10:09:31.518+02:00Asshole in Range Rover<p>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgkbfThArTV-jr7nSRoL7XxiwP-FBPnMuTUuYmgBS3qxVF6yz8z3kKyCZ_kGh3_9JLlMkFqTLew2bBcxQY89FdTmr3k7FMFXALjyRFysLw4_s3tKbEdnhd2S7uob_YJ_aDDG4SjaH2_Kl7O/s1600/3bbeac92e09e4c51abf353f27326ef96.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgkbfThArTV-jr7nSRoL7XxiwP-FBPnMuTUuYmgBS3qxVF6yz8z3kKyCZ_kGh3_9JLlMkFqTLew2bBcxQY89FdTmr3k7FMFXALjyRFysLw4_s3tKbEdnhd2S7uob_YJ_aDDG4SjaH2_Kl7O/s1600/3bbeac92e09e4c51abf353f27326ef96.jpg" /></a>
Сегодня красиво наказал энурезника очередного на белом коне. Стоял в левом ряду на Автозаводской (по 3 полосы в каждую сторону) первым, смотрю в зеркало - а там очередное хуйло, обгоняя аж 3-4 автомобиля, прет по встречке. Я дал по газам и не позволил белому Range Rover'у стать перед собой. А на другом конце перекрестка в том же левом ряду первыми стояли угадайте кто? Правильно - пацаны с красными и синими мигалками. Результат - быдло к тротуару. Энурез нужно лечить, господа, ЛЕЧИТЬ.
</p>Anonymoushttp://www.blogger.com/profile/14003906116514303118noreply@blogger.com0tag:blogger.com,1999:blog-264654303037912405.post-72555612905302146782013-08-29T15:49:00.002+03:002013-08-29T15:49:22.893+03:00Cruel DateTime vs serialization<p>
Недавно столкнулся с проблемой сериализации <code>DateTime</code>. protobuf-net, как и <code>BinaryFormatter</code> не сохраняют тип даты, перечисление <a href="http://msdn.microsoft.com/ru-ru/library/shx7s921.aspx" target="_blank"><code>DateTimeKind</code></a>. В результате после чтения из архива тип даты становится <code>Unspecified</code>. Вот выдержка из исходного кода структуры <code>DateTime</code>:
</p>
<pre>
// This value type represents a date and time. Every DateTime
// object has a private field (Ticks) of type Int64 that stores the
// date and time as the number of 100 nanosecond intervals since
// 12:00 AM January 1, year 1 A.D. in the proleptic Gregorian Calendar.
//
// Starting from V2.0, DateTime also stored some context about its time
// zone in the form of a 3-state value representing Unspecified, Utc or
// Local. This is stored in the two top bits of the 64-bit numeric value
// with the remainder of the bits storing the tick count. This information
// is only used during time zone conversions and is not part of the
// identity of the DateTime. Thus, operations like Compare and Equals
// ignore this state. This is to stay compatible with earlier behavior
// and performance characteristics and to avoid forcing people into dealing
// with the effects of daylight savings. Note, that this has little effect
// on how the DateTime works except in a context where its specific time
// zone is needed, such as during conversions and some parsing and formatting
// cases.
//
// There is also 4th state stored that is a special type of Local value that
// is used to avoid data loss when round-tripping between local and UTC time.
// See below for more information on this 4th state, although it is
// effectively hidden from most users, who just see the 3-state DateTimeKind
// enumeration.
//
// For compatability, <em>DateTime does not serialize the Kind data when used in
// binary serialization</em>.
//
// For a description of various calendar issues, look at
//
// Calendar Studies web site, at
// http://serendipity.nofadz.com/hermetic/cal_stud.htm.
//
//
[StructLayout(LayoutKind.Auto)]
[Serializable]
public struct DateTime : IComparable, IFormattable, IConvertible, ISerializable, IComparable<DateTime>,IEquatable<DateTime> {
</pre>
<p>
Как видно из описания, дата представляет собой число типа <code>Int64</code> в котором хранится количество 100нс интервалов от начала времен - <q>12:00 AM January 1, year 1 A.D. in the proleptic Gregorian Calendar</q>. А вот до какой точки - здесь уже интересней. В случае <code>DateTimeKind.Utc</code> - до Гринвича, <code>DateTimeKind.Local</code> - до времени в локальной для операционной системы\программы\потока зоне. И последнее значение - <code>DateTimeKind.Unspecified</code>, до куда - неизвестно.
</p>
<p>
На что влияет тип даты? В первую очередь на методы <code>ToLocalTime</code> и <code>ToUniversalTime</code>, потом уже и на форматирующие методы. Самое неприятное происходит при вызове этих двух методов для дат с типом <code>DateTimeKind.Unspecified</code> - <code>ToLocalTime</code> считает, что дата имеет тип <code>DateTimeKind.Utc</code>, а <code>ToUniversalTime</code> - что тип <code>DateTimeKind.Local</code>. Логично, правда? В результате если сериализировать <code>DateTime.UtcNow</code>, вычитать его обратно и преобразовать в <code>DateTimeKind.Utc</code> методом <code>ToUniversalTime</code> - получаем сдвиг на временную зону. При этом <code>ToLocalTime</code> вернет правильный результат.
</p>
<p>
Обойти это недоразумение можно с помощью статического метода <code>DateTime.SpecifyKind</code>.
</p>
<pre>
using System;
namespace CSharpLanguageInv
{
internal class Program
{
private static void Main(string[] args)
{
var now = DateTime.UtcNow;
var unspecified = DateTime.SpecifyKind(now, DateTimeKind.Unspecified);
var localTime = unspecified.ToLocalTime();
var universalTime = unspecified.ToUniversalTime();
Console.WriteLine("Now = " + now/*.Ticks*/);
Console.WriteLine("unspecified = " + unspecified/*.Ticks*/);
Console.WriteLine("localTime = " + localTime/*.Ticks*/);
Console.WriteLine("universalTime = " + universalTime/*.Ticks*/);
Console.WriteLine("Now - unspecified = " + (now/*.Ticks*/ - unspecified/*.Ticks*/));
Console.ReadLine();
}
}
}
</pre>Anonymoushttp://www.blogger.com/profile/14003906116514303118noreply@blogger.com1tag:blogger.com,1999:blog-264654303037912405.post-11493032565846990692013-08-27T13:30:00.001+03:002013-08-27T13:30:35.718+03:00Razer DeathStalker (not Ultimate)<p>Завидуйте, нищеброды! Американская раскладка, с не кастрированным левым шифтом и нормальным вводом! USA! USA! USA!</p>
<div class="separator" style="clear: both; text-align: center;"><a href="http://assets.razerzone.com/eeimages/products/771/razer-dstalk-gallery-3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://assets.razerzone.com/eeimages/products/771/razer-dstalk-gallery-3.png" /></a></div>Anonymoushttp://www.blogger.com/profile/14003906116514303118noreply@blogger.com0tag:blogger.com,1999:blog-264654303037912405.post-83960079383209484622013-08-12T21:23:00.001+03:002013-08-12T21:23:39.113+03:00Maelstorm- Wild Dances<iframe allowfullscreen="" frameborder="0" height="270" src="//www.youtube.com/embed/hIn9BcT3RVg" width="480"></iframe>Anonymoushttp://www.blogger.com/profile/14003906116514303118noreply@blogger.com0tag:blogger.com,1999:blog-264654303037912405.post-77988755059281806432013-07-26T13:48:00.001+03:002013-07-26T13:48:53.834+03:00Windows Kernel Source code<a href="http://www.awarenetwork.org/home/iqlord/other/wrk.rar">Windows Research Kernel sources.</a>
Anonymoushttp://www.blogger.com/profile/14003906116514303118noreply@blogger.com0tag:blogger.com,1999:blog-264654303037912405.post-41866156740394897832013-07-12T11:44:00.001+03:002013-07-12T11:44:17.334+03:00Freddie Mercury - Living On My Own<iframe allowfullscreen="" frameborder="0" height="344" src="//www.youtube.com/embed/qqZYG58u3S4" width="459"></iframe>Anonymoushttp://www.blogger.com/profile/14003906116514303118noreply@blogger.com0tag:blogger.com,1999:blog-264654303037912405.post-81419854693817887502013-07-09T14:53:00.000+03:002013-07-09T14:57:50.154+03:00A story of orphaned ReaderWriterLockSlim<p>Recently I got 2 dumps of a resource intensive process. The customer complained about hangs in web UI so the application had been killed and restarted numerous
times. Quick WinDbg analysis spotted thousands of working threads in the pool:</p>
<pre>
0:000> !ThreadPool
CPU utilization: 6%
Worker Thread: <strong>Total: 6304 Running: 6303</strong> Idle: 1 MaxLimit: 12000 MinLimit: 24
Work Request in Queue: 0
--------------------------------------
Number of Timers: 2
--------------------------------------
Completion Port Thread:Total: 2 Free: 1 MaxFree: 48 CurrentLimit: 1 MaxLimit: 12000 MinLimit: 24
</pre>
<p>Most of the threads wait for <code>ReaderWriterLockSlim</code> read lock on <code>ManualResetEvent</code> instance:</p>
<pre>
System.Threading.WaitHandle.WaitOneNative(System.Runtime.InteropServices.SafeHandle, UInt32, Boolean, Boolean)
System.Threading.WaitHandle.InternalWaitOne(System.Runtime.InteropServices.SafeHandle, Int64, Boolean, Boolean)
System.Threading.ReaderWriterLockSlim.WaitOnEvent(System.Threading.EventWaitHandle, UInt32 ByRef, TimeoutTracker)
System.Threading.ReaderWriterLockSlim.TryEnterReadLockCore(TimeoutTracker)
System.Threading.ReaderWriterLockSlim.TryEnterReadLock(TimeoutTracker)
</pre>
<p>One thread was waiting for write lock on the same object. No other stacks observed executing while holding the lock, all lock usages seemed proper:</p>
<pre>
s.EnterXXXLock();
try
{
// Do the job
}
finally
{
s.ExitXXXLock();
}
</pre>
<p>Yet the process is fucked up. What the hell is wrong here? Well, sometimes things get very complicated...</p>
<p>Lets take a look on reader writer lock instance:</p>
<pre>
0:3444> !do 0x0000000001affe60
Name: System.Threading.ReaderWriterLockSlim
MethodTable: 000007f87a91c1a8
EEClass: 000007f87a639448
Size: 96(0x60) bytes
File: C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Core\v4.0_4.0.0.0__b77a5c561934e089\System.Core.dll
Fields:
MT Field Offset Type VT Attr Value Name
000007f8802fc7b8 4000755 50 System.Boolean 1 instance 1 fIsReentrant
000007f8802fdc90 4000756 30 System.Int32 1 instance 0 myLock
000007f8802f1ed0 4000757 34 System.UInt32 1 instance 1 numWriteWaiters
000007f8802f1ed0 4000758 38 System.UInt32 1 instance 6293 numReadWaiters
000007f8802f1ed0 4000759 3c System.UInt32 1 instance 0 numWriteUpgradeWaiters
000007f8802f1ed0 400075a 40 System.UInt32 1 instance 0 numUpgradeWaiters
000007f8802fc7b8 400075b 51 System.Boolean 1 instance 0 fNoWaiters
000007f8802fdc90 400075c 44 System.Int32 1 instance -1 upgradeLockOwnerId
000007f8802fdc90 400075d 48 System.Int32 1 instance -1 writeLockOwnerId
000007f8802f8d00 400075e 8 ...g.EventWaitHandle 0 instance 00000000f8e8f9c0 writeEvent
000007f8802f8d00 400075f 10 ...g.EventWaitHandle 0 instance 00000000fa23f040 readEvent
000007f8802f8d00 4000760 18 ...g.EventWaitHandle 0 instance 0000000000000000 upgradeEvent
000007f8802f8d00 4000761 20 ...g.EventWaitHandle 0 instance 0000000000000000 waitUpgradeEvent
000007f88030ff60 4000763 28 System.Int64 1 instance 9 lockID
000007f8802fc7b8 4000765 52 System.Boolean 1 instance 0 fUpgradeThreadHoldingRead
000007f8802f1ed0 4000766 4c System.UInt32 1 instance <strong>1073741824</strong> owners
000007f8802fc7b8 4000767 53 System.Boolean 1 instance 0 fDisposed
000007f88030ff60 4000762 408 System.Int64 1 static 17381 s_nextLockID
000007f87a9399f0 4000764 8 ...ReaderWriterCount 0 TLstatic t_rwc
>> Thread:Value c18:0000000001917410 d18:00000000025a51c8 e54:000000000245d5f0 e90:0000000000000000 e20:00000000f90a6ce8 [>6000 more values]
</pre>
<p>The most valuable information is the <code>owners</code> field:</p>
<pre>
0:000> ? 0n1073741824
Evaluate expression: 1073741824 = 00000000`40000000
</pre>
<p>And heres what it means:</p>
<pre>
//The uint, that contains info like if the writer lock is held, num of
//readers etc.
uint owners;
//Various R/W masks
//Note:
//The Uint is divided as follows:
//
//Writer-Owned Waiting-Writers Waiting Upgraders Num-REaders
// 31 30 29 28.......0
//
//Dividing the uint, allows to vastly simplify logic for checking if a
//reader should go in etc. Setting the writer bit, will automatically
//make the value of the uint much larger than the max num of readers
//allowed, thus causing the check for max_readers to fail.
private const uint WRITER_HELD = 0x80000000;
private const uint <strong>WAITING_WRITERS = 0x40000000</strong>;
private const uint WAITING_UPGRADER = 0x20000000;
</pre>
<p>So, we are waiting for writers. Hold on, there are no writers! The lock is not held. Conslusion - the lock state is corrupted and could never recover. This is
called orphaned lock.</p>
<p>The only thing (I am aware of) might have caused the orphan - asynchronous thread aborts. If a thread is interrupted while taking a lock via <code>[Try]EnterXXXLock</code>
method - we might come into described problem since those methods are not atomic. In my case thread aborts are triggered by WCF runtime (or perhaps Http runtime,
it doesn't matter).</p>
<p>Heres a simple code to simulate the situation:</p>
<pre style="font-family: Consolas; font-size: 14; color: black; background: white;"><span style="color: navy;">using</span> <span style="font-weight: bold; color: navy;">System</span>;
<span style="color: navy;">using</span> <span style="font-weight: bold; color: navy;">System</span><span style="color: #800040;">.</span><span style="font-weight: bold; color: navy;">Threading</span>;
<span style="color: navy;">namespace</span> <span style="font-weight: bold; color: navy;">CLRInv</span>
{
<span style="color: navy;">internal</span> <span style="color: navy;">class</span> <span style="color: blueviolet;">Program</span>
{
<span style="color: navy;">private</span> <span style="color: navy;">static</span> <span style="color: navy;">readonly</span> <span style="color: blueviolet;">ReaderWriterLockSlim</span> <span style="color: lightseagreen;">rwl</span> <span style="color: #800040;">=</span> <span style="color: navy;">new</span> <span style="color: blueviolet;">ReaderWriterLockSlim</span>(<span style="color: olive;">LockRecursionPolicy</span><span style="color: #800040;">.</span><span style="font-weight: bold; color: purple;">SupportsRecursion</span>);
<span style="color: navy;">private</span> <span style="color: navy;">static</span> <span style="color: navy;">void</span> <span style="color: crimson;">Main</span>(<span style="color: navy;">string</span>[] <span style="color: saddlebrown;">args</span>)
{
<span style="color: lightseagreen;">rwl</span><span style="color: #800040;">.</span><span style="color: crimson;">EnterReadLock</span>();
<span style="color: navy;">do</span> {
<span style="color: lightseagreen;">rwl</span><span style="color: #800040;">.</span><span style="color: crimson;">ExitReadLock</span>();
<span style="color: navy;">var</span> <span style="color: lightslategray;">reader</span> <span style="color: #800040;">=</span> <span style="color: navy;">new</span> <span style="color: blueviolet;">Thread</span>(<span style="color: crimson;">UseLockForRead</span>);
<span style="color: navy;">var</span> <span style="color: lightslategray;">writer</span> <span style="color: #800040;">=</span> <span style="color: navy;">new</span> <span style="color: blueviolet;">Thread</span>(<span style="color: crimson;">UseLockForWrite</span>);
<span style="color: lightslategray;">reader</span><span style="color: #800040;">.</span><span style="color: crimson;">Start</span>();
<span style="color: lightslategray;">writer</span><span style="color: #800040;">.</span><span style="color: crimson;">Start</span>();
<span style="color: blueviolet;">Thread</span><span style="color: #800040;">.</span><span style="color: crimson;">Sleep</span>(<span style="color: #008040;">TimeSpan</span><span style="color: #800040;">.</span><span style="color: crimson;">FromSeconds</span>(<span style="color: darkgreen;">2</span>));
<span style="color: lightslategray;">writer</span><span style="color: #800040;">.</span><span style="color: crimson;">Abort</span>();
<span style="color: lightslategray;">reader</span><span style="color: #800040;">.</span><span style="color: crimson;">Abort</span>();
<span style="color: lightslategray;">reader</span><span style="color: #800040;">.</span><span style="color: crimson;">Join</span>();
<span style="color: lightslategray;">writer</span><span style="color: #800040;">.</span><span style="color: crimson;">Join</span>();
}
<span style="color: navy;">while</span> (<span style="color: lightseagreen;">rwl</span><span style="color: #800040;">.</span><span style="color: crimson;">TryEnterReadLock</span>(<span style="color: #008040;">TimeSpan</span><span style="color: #800040;">.</span><span style="color: crimson;">FromSeconds</span>(<span style="color: darkgreen;">10</span>)));
<span style="font-weight: bold; color: blueviolet;">Console</span><span style="color: #800040;">.</span><span style="color: crimson;">WriteLine</span>(<span style="color: magenta;">"Gotcha!"</span>);
<span style="color: #3e5ebd;">// Forever young</span>
<span style="color: lightseagreen;">rwl</span><span style="color: #800040;">.</span><span style="color: crimson;">EnterWriteLock</span>();
}
<span style="color: navy;">private</span> <span style="color: navy;">static</span> <span style="color: navy;">void</span> <span style="color: crimson;">UseLockForRead</span>()
{
<span style="color: navy;">try</span> {
<span style="color: navy;">for</span> (;;) {
<span style="color: lightseagreen;">rwl</span><span style="color: #800040;">.</span><span style="color: crimson;">EnterReadLock</span>();
<span style="color: navy;">try</span> {
}
<span style="color: navy;">finally</span> {
<span style="color: lightseagreen;">rwl</span><span style="color: #800040;">.</span><span style="color: crimson;">ExitReadLock</span>();
}
}
}
<span style="color: navy;">catch</span> (<span style="color: blueviolet;">ThreadAbortException</span>) {
}
}
<span style="color: navy;">private</span> <span style="color: navy;">static</span> <span style="color: navy;">void</span> <span style="color: crimson;">UseLockForWrite</span>()
{
<span style="color: navy;">try</span> {
<span style="color: navy;">for</span> (;;) {
<span style="color: lightseagreen;">rwl</span><span style="color: #800040;">.</span><span style="color: crimson;">EnterWriteLock</span>();
<span style="color: navy;">try</span> {
}
<span style="color: navy;">finally</span> {
<span style="color: lightseagreen;">rwl</span><span style="color: #800040;">.</span><span style="color: crimson;">ExitWriteLock</span>();
}
}
}
<span style="color: navy;">catch</span> (<span style="color: blueviolet;">ThreadAbortException</span>) {
}
}
}
}
</pre>
<p>The conclusion is not very optimistic - you can't use slim locks the way you normally use em if your application experiences timeouts and consequent thread aborts.
Does this mean slim locks should be banned? Well, no. You just need to ensure special constructions are used to take and release locks.</p>
<p>First of all we need to prevent async aborts while executing <code>[Try]EnterXXXLock</code>. How to do that? You must take the lock inside so called <q>protected
region</q>. <a href="http://msdn.microsoft.com/en-us/library/ty8d3wta.aspx" target="_blank" title="Thread.Abort Method">Here</a> they mention <q>a protected region
of code, such as a catch block, finally block, or constrained execution region</q>. This basically means <code>ThreadAbortExeption</code> can't be thrown asynchronously
while executing <code>except</code> and <code>finally</code> blocks of a <code>try</code> statement. So our <code>[Try]EnterXXXLock</code> should be wrapped like
this:</p>
<pre>
try {} finally { rw.EnterXXXLock(); }
</pre>
<p>Weird? No, if you have .NET BCL source code. There are tonns of empty <code>try</code> blocks with excessive comments:</p>
<pre>
// prevent ThreadAbort while updating state
try { }
finally
{
...
}
</pre>
<p>Proper slim lock usage turns out to be the following construction:</p>
<pre style="font-family: Consolas; font-size: 14; color: black; background: white;"><span style="color: navy;">var</span> <span style="color: lightslategray;">lockIsHeld</span> <span style="color: #800040;">=</span> <span style="color: navy;">false</span>;
<span style="color: navy;">try</span> {
<span style="color: navy;">try</span> {
}
<span style="color: navy;">finally</span> {
<span style="color: lightseagreen;">rwl</span><span style="color: #800040;">.</span><span style="color: crimson;">EnterReadLock</span>();
<span style="color: lightslategray;">lockIsHeld</span> <span style="color: #800040;">=</span> <span style="color: navy;">true</span>;
}
<span style="color: #3e5ebd;">// Do work here</span>
}
<span style="color: navy;">finally</span> {
<span style="color: navy;">if</span> (<span style="color: lightslategray;">lockIsHeld</span>) {
<span style="color: lightseagreen;">rwl</span><span style="color: #800040;">.</span><span style="color: crimson;">ExitReadLock</span>();
}
}</pre>
<p>Asynchronous <code>ThreadAbortException</code> is thrown either before lock is held or after lock is held making finally unlock the object if it has been locked.</p>
<p>Two things I havent studied yet - is it possible to observe the following situation:</p>
<pre style="font-family: Consolas; font-size: 14; color: black; background: white;"><span style="color: navy;">try</span> {
<span style="color: #3e5ebd;">// <-- Could it happen here, before finally block is run but after try has opened fault clause region?</span>
<span style="color: navy;">try</span> {
}
<span style="color: navy;">finally</span> {
<span style="color: #3e5ebd;">// Lock</span>
}
<span style="color: #3e5ebd;">// Use resource</span>
}
<span style="color: navy;">finally</span> {
<span style="color: #3e5ebd;">// Unlock</span>
}</pre>
<p>Thats why I used that condition flag to ensure the lock is held.</p>
<p>And the second one:</p>
<pre style="font-family: Consolas; font-size: 14; color: black; background: white;"><span style="color: navy;">try</span> {
}
<span style="color: navy;">finally</span> {
<span style="color: #3e5ebd;">// Lock</span>
}
<span style="color: navy;">try</span> {
<span style="color: #3e5ebd;">// Use resource</span>
}
<span style="color: navy;">finally</span> {
<span style="color: #3e5ebd;">// Unlock</span>
}</pre>
<p>Is this one safe? Probably <a href="http://www.bluebytesoftware.com/blog/2007/01/30/MonitorEnterThreadAbortsAndOrphanedLocks.aspx" target="_blank">yes</a>.</p>
<p class="warning">The bottom line is <strong>know your runtime environment, don't use new features cause they are cool or Mr. Jeff has fresh stuff in his brand new book you love so much</strong>. Or hire a professional like me [:-D].</p>
Anonymoushttp://www.blogger.com/profile/14003906116514303118noreply@blogger.com12tag:blogger.com,1999:blog-264654303037912405.post-63889937617703432722013-07-03T20:56:00.001+03:002013-07-03T20:56:19.918+03:00Mortal Kombat 9 Komplete PC edition in steam<p>
<a href="http://www.cinemablend.com/images/gallery/s57168/Warner_Bros_Entertainment_13727149111485.jpg" imageanchor="1" ><img border="0" src="http://www.cinemablend.com/images/gallery/s57168/Warner_Bros_Entertainment_13727149111485.jpg" /></a>
</p>Anonymoushttp://www.blogger.com/profile/14003906116514303118noreply@blogger.com0tag:blogger.com,1999:blog-264654303037912405.post-23275572731064056472013-06-26T08:43:00.001+03:002013-06-26T08:43:30.925+03:00 Oracle Database 12c Release 1<p>
<a href="http://www.oracle.com/technetwork/database/enterprise-edition/downloads/index.html">Available for download!</a> Linux/Solaris only, as always...
</p>Anonymoushttp://www.blogger.com/profile/14003906116514303118noreply@blogger.com1