Is AI Scraping Your PDFs? How to Protect Your Documents with TDMRep
AI can read, scrape, and train on the data inside your PDFs - unless you tell them not to. In this conversation with Datalogics Support Rep Alec Goldstein, we break down TDM Reservation Protocol (TDMRep): what it is, why it matters, and how to add it to your PDF metadata using the Adobe PDF Library SDK to reserve your data rights.
Video Transcript:
Lindsey: I am Lindsey and I'm here with Alec today to chat about AI and how to prevent AI from scraping PDFs with something called TDM rep. I I thought it now with AI accelerating as quickly as it is that now would be a good time to kind of chat about this in a little bit more detail because as everybody knows AI is everywhere and is kind of taking over. I wanted to chat with you today about what people need to know about AI scraping the PDFs.
Alec: Of course.
Lindsey: Yeah, so before we get into the questions, would you mind just briefly explaining your role at Datalogics?
Alec: Yes, I'm a support rep. So people tell me about problems they're having either you know, they could be bugs or they just have questions on how to best optimize their workflows or you know, some very specific technical questions and it's my job to take in those requests and either just answer on the spot or pass it off to engineering if I think it needs more deep expertise or if it requires a bug fix.
Lindsey: Okay, so let's get into the first question. So the first question I have for you is can you explain kind of in a nutshell how AI can scrape PDFs and what it can do with the data within those PDFs?
Alec: Yeah, so essentially AI and when we say AI we're particularly talking about large language models such as chat GPT and the like. They will consume entire documents during the training process and the actors behind these models and their training processes you know, they cast a very wide net in the training. So you'd be surprised what documents they'd be able to find if it's publicly accessible of course and once that document is consumed by the model the data in that document can be used however the actor sees fit.
Lindsey: Okay, and next question is can you kind of explain so we're going to be talking about TDM rep which stands for text and data mining protocol. Is that right?
Alec: TDM Rep stands for text and data mining reservation protocol.
Lindsey: Got it. Okay.
Alec: Quite a mouthful.
Lindsey: Yes, for sure. Do you mind explaining what that is in practical terms?
Alec: Yeah, it's essentially just a means by which we can one, express rights reservation over content, and two, provide information on how to legally obtain permission to use that content if desired by the author.
Lindsey: Okay.
Alec: TDM Rep is a protocol created by the W3C, and it's essentially just metadata. That's all it is.
Lindsey: Oh, okay. So, using our software, how can TDM Rep be implemented into it?
Alec: Yep, it's quite simple. So, TDM Rep is just a block of metadata that goes into a PDF. And so, AP Level does this just by using our metadata insertion functions. For example, in the .NET interface, we have set XMP metadata property in the document class.
Lindsey: Mhm, okay.
Alec: So, essentially, you're just inserting the TDM Rep specific metadata into the PDF, and when the AI model happens upon this PDF, it may see this, which expresses rights reservation.
Lindsey: Got you. Okay. And for people who want to see, we have code samples that show this, right?
Alec: Yes, we do. Yeah, and I mentioned that in the KB article.
Lindsey: Amazing. So, as far as being concerned with AI scraping PDFs, who what kind of industries, what kind of people should be concerned about AI scraping their PDFs, and um like, how should they kind of be aware of of what they need to do, or if they if they might be implicated in having their their PDFs scraped by AI?
Alec: Yeah, in my opinion, I think that any person or institution with one or more publicly accessible documents should be concerned about AI consuming those documents, but especially if those documents contain any sensitive information such as personal info or industry secrets.
Lindsey: Mhm. Yeah, so things like legal, especially, or finance, financial information, health...
Alec: Could be legal things, financial, health care. I mean, obviously, you know, that stuff shouldn't be public anyway, but if it were...
Lindsey: Yeah, but in case...
Alec: ...and AI AI finds it, AI can do whatever it wants with it.
Lindsey: Yeah, okay. That makes sense.
Alec: So, a note I want to leave on that is that while TDM rep cannot physically prevent public documents from being consumed, it provides a framework, again, by which we express the rights over the documents should an LLM trainer find them.
It's less of a gate, more of a sign on your PDF's front door that says, 'Please knock before entering.'
Lindsey: Got it. So, it's like it's basically giving them permission.
Alec: Yes, or not.
Lindsey: Or not, right. Okay. And then finally, if somebody would rather allow AI to scrape their documents for whatever reason, is there a way to do that as well?
Alec: Yes, the process would be exactly the same. You would just set the value differently in the uh TDM rep metadata to allow the scraping.
Lindsey: Got you. Okay.
Alec: Or, you know, more specifically to express that uh certain rights are not reserved and that there is no uh contact or payment or whatever required to uh access that data.
Lindsey: Yeah, there's no barrier in the way.
Alec: Yes. Yeah, and uh as I said, everything is expressed in more detail in the knowledge base article that I've written.
Lindsey: Excellent. I'm going to, speaking of that, I'm going to, in the description of this video, I'm going to add the KB article as well as the article that we uh put together a while back for more resources if people want to learn more.
Alec: As always, if you have any other questions, contact Datalogics support, tech_support@datalogics.com, and either I or one of my colleagues will answer it.
Lindsey: Thank you so much for your time. And like I said, I'm going to link those articles in the description here if you want to learn more about it.
Links for reference:
Adobe PDF Library Info & Free Trial