If the origins of Moyse's Hall are mysterious, so too is its early history. There is a tradition that it was a tavern in the medieval period. There is both historical and architectural evidence that at least part of it was an inn in later years, but the evidence for this early period is slight. The earliest reference to Moyse's Hall occurs in 1328 and although it suggests that Moyse's Hall may have been a tavern, it does not prove it. It is a very colourful story, however, and is worth quoting for its own sake.

In 1327, the long simmering disputes between the abbey and the townsfolk erupted into a riot. The townsfolk entered the abbey, damaged many of the buildings and looted them. Sporadic outbursts of violence continued into the next year. A contemporary tract called 'Depraedatio Abbatiae Sancti Edmundi' states that 'After this about midnight on St Helen's Day, in the same year (August 18th, 1328) came Thomas de Thornham with many fugitives and outlaws into the town of St Edmund, and by force seized the keys of all the gates of the town, and none of the townsmen saying them nay, hurried off to breakfast at Moyse's Hall, and on their way killed Roger Peasenhall, a servant of the Abbey. And the men of the town being full of joy at their coming, celebrated this breakfast by the many gifts they made towards it. But one thing happened there which might have struck fear into them. For when a woman out of Cooks' Street had come to take the price of the meal she had sold them she looked into the cellar close to them, and saw a most horrible devil, as though he were writing. And all of a tremble at his horrible countenance, she drew back the foot which she had put across the threshold, but not without punishment, for instantly, that foot was roasted by infernal fire, and fell rotting from her body.'

In the late 15th and 16th centuries, there are several references to Moyse's Hall being owned by a family called King. For example, in 1514 a merchant called Richard King bequeathed to his wife Cecily 'my place and tenement that I dwell in called Moyse's Hall'