If Edward V had immediately responded with “who are you”, or words to that effect, then the cover would have been blown.
There have been numerous pretenders claiming to be Richard, Duke of York. The most notable was Perkin Warbeck. Warbeck was a thorn in the side of Henry VII for years. Warbeck had first appeared in Ireland claiming to be the youngest of the two princes and he was immediately embraced as the heir to the house of York. Warbeck even managed to convince other European heads of his claim, although their support may simply have been to agitate the King of England. Margaret of Burgundy supported his claim (she was the sister of Edward IV and Richard III). Warbeck's appearance from portraits does show a vague likeness to Edward IV. His knowledge of the English court and his “family history” was without fault. If he were not York, then he was a highly convincing impersonator.
Warbeck never claimed that his mother had switched him before getting him away. Instead, Warbeck said that he was shown pity by the murderers of his brother who sent him overseas. This would seem highly unlikely. Men who would murder one boy are hardly likely to show pity on another a few moments afterwards. The changeling theory would have been significantly more plausible.
After his capture, Warbeck confessed that he wasn't Edward IV's son. Henry VII had copies of the confession printed and distributed to further hammer home this fact. Of course, you might say that once captured you would confess to anything to avoid torture and a painful death, and so the confession is worthless.
It is interesting that all of the pretenders claimed to be the Duke of York and not Edward V. This was because this changeling theory was one that was often talked about.
The other point in favour of this theory is the fact that Elizabeth Woodville gave up the Duke of York to Gloucester. Some Ricardians suggest this was because she believed that he was in no danger. If this were the case then why were the Woodvilles in sanctuary at all? Either Woodville knew that the boy she was giving up wasn't her son or she knew that the troops would enter and take him anyway.
There isn't significant conclusive evidence either way. However, once again I am going with my gut feel and suggesting that a changeling was never offered. Trying to bluff Gloucester that the changeling was his nephew would have simply been too tall an order. The boy that was taken away from the abbey that day was indeed Richard of Shrewsbury, Duke of York. Once again Elizabeth Woodville was practicing self-preservation; she was giving up one child to protect the rest of the clan. If something did happen to her sons, then she would still be in possession of the next heir, her own daughter, Elizabeth of York.
Other theories regarding the fate of the princes
Of course there are other theories about the final fate of the princes. Some suggest that the princes were removed overseas, others that they were sent away to the country.
One theory is that the boys were sent up to Sheriff Hutton, in the heart of Richard's power base. This theory seems to have its basis in an entry in the household accounts about clothing for the “Lord Bastard.” However, this entry is more likely to refer to Richard III's own openly acknowledged illegitimate son, John of Gloucester rather than Edward V. However, it is not outside the realms of possibility that this entry could refer to Edward.
There is also an interesting theory that there is a hidden message in the portrait of the family of Sir Thomas More. The theory basically states that the man in the rear of the portrait standing underneath the fleur-de-ley canopy is actually Richard, Duke of York. In reality the man depicted is Dr. John Clement, a man who was brought up in More's household. The theory comes from the fact that the portrait has the words “ John, the rightful heir” above the figure of Clement. We don't know much about Clement, but