Yes, you can be charged.
Whatever happens now, you aren't required to speak with the cops, or answer any questions (EXCEPT: name, date of birth, address - if arrested).
If they come again, politely decline the opportunity to chat with them, except name, address, etc...
If they arrest you, same drill. Don't get loud, don't try to run, don't try to explain: simply politely decline to answer any questions. You need not explain why, just keep politely saying, "sorry, I don't want to talk", or "I have nothing to say".
They'll try all sorts of tricks, just stand your ground, don't talk, don't be smart or act tough.
Anything you say can and will be used against your interests in their court of law.