The Treasury Department is investigating whether the crypto exchange allowed users in Iran to buy and sell digital tokens, said people with knowledge of the matter.