I'll tell you what...
I figured it out yesterday. When I used mpiexec it gave me a warning says that it cannot connect because user-name and password (of the remote computer) are incorrect. Even though I had them in the credential store. So I created another account with the same user-name and password like the hosting computer - and it worked.
It seems like it doesn't try all the user-name & password pairs - just the first one.