How to Avoid Duplicates in Database?
-
Every 10 minutes I am scraping a news website for new articles and save the Title + Content into a Database. The problem is that instead of ignoring the aready existing Titles/Content it just adds the same entries again below. So how can I avoid that?
-
good question. out of curiosity i checked databases and tried to check with javascript .match or .IndexOf for a string but it either resulted false or error when string was there.
I also took list from database and checked with list contains option to see if its there, but it returned false
-
Ok I managed to do it this way:
be careful for notreuse option
-
Thank you! Will try it now
-
I have the same problem.
Unfortunately I don't see how the string matches regex function can help.
It allows to check a variable, but I don't see how I could check all the records in a table?Any ideas or other solutions?