{"id":86,"date":"2023-02-02T15:28:28","date_gmt":"2023-02-02T20:28:28","guid":{"rendered":"https:\/\/carleton.ca\/xlab\/?p=86"},"modified":"2023-05-24T14:43:26","modified_gmt":"2023-05-24T18:43:26","slug":"knowledge-graphs-and-gpt-3","status":"publish","type":"post","link":"https:\/\/carleton.ca\/xlab\/2023\/knowledge-graphs-and-gpt-3\/","title":{"rendered":"Knowledge Graphs and GPT-3"},"content":{"rendered":"

One of the things we’re working on in our group are ways to extract structured knowledge from lots of unstructured text. In an article coming out soon in Advances in Archaeological Practice<\/a>, we demonstrate some of the things that we can accomplish when we have statements about a domain (in this case, the antiquities trade) arranged as subject, verb, object statements and then knit into a graph. The graph can be turned into a series of numerical vectors (think of each vector as a line shooting out in a different direction in a multidimensional space; call this a knowledge graph embedding model), and then we can make predictions about things we don’t<\/em> know based on measuring the distances between those vectors. (We used a package called Ampligraph<\/a>)<\/p>\n

\"\"<\/p>\n

\"\"<\/p>\n

\"\"<\/p>\n

(illustrations of that process from Ampligraph)<\/p>\n

But getting the knowledge out<\/em> in the first place, from the text, so we can do something with it… that’s the hard part. We manually annotated 129 articles by hand (using a tool that used XML to indicate the relationship between the annotations and the text) and then Chantal wrote a script to turn those annotations into a CSV file. That took us several months. Surely there’s got to be a faster way?<\/p>\n

We started playing with GPT3<\/a>, the large language model that some fear will ruin education, others fear will put people out of jobs, and still some others figure will create jobs that don’t exist yet. We just wanted to know if it could identify subject, verbs, and objects from an article, selecting the most important bits. It can, and yes, it does feel spooky when it works. But it can make mistakes.<\/p>\n

We adapted this tutorial\u00a0<\/a>so that we could pass one cell of data from a spreadsheet through the OpenAI API. Each cell contains the full text of an article we want to analyze. The script prepends a prompt to that text, and the prompt steers the generative power of the model in a particular direction. Having seen untold numbers of examples of internet texts that summarize movies, books, whatever, if you prompted the model with ‘Summarize this article’, you’d duly get a three or four line summary that captures the main point. What’s more, if you told it, ‘Sum<\/span>mar<\/span>ize<\/span> the<\/span> text<\/span>,<\/span> but<\/span> do<\/span> it<\/span> in<\/span> the<\/span> style<\/span> of<\/span> a<\/span> 1940<\/span>s<\/span> gang<\/span>ster<\/span> film<\/span> where<\/span> two<\/span> gang<\/span>sters<\/span> are<\/span> talking<\/span>‘ and then gave it the text of this article about the Sarpedon Krater,<\/a> you’d get:<\/span><\/p>\n

G<\/span>ang<\/span>ster<\/span> 1<\/span>:<\/span> “<\/span>Did<\/span> ya<\/span> hear<\/span> ‘<\/span>b<\/span>out<\/span> that<\/span> k<\/span>r<\/span>ater<\/span> they<\/span> got<\/span> at<\/span> the<\/span> Metropolitan<\/span> Museum<\/span>?<\/span> Word<\/span> is<\/span> it<\/span> was<\/span> made<\/span> in<\/span> Athens<\/span> like<\/span> 500<\/span> years<\/span> ago<\/span>.<\/span> They<\/span> paid<\/span> a<\/span> million<\/span> bucks<\/span> for<\/span> it<\/span>!<\/span><\/p>\n

G<\/span>ang<\/span>ster<\/span> 2<\/span>:<\/span> Yeah<\/span>,<\/span> I<\/span> heard<\/span>.<\/span> Rum<\/span>or<\/span> has<\/span> it<\/span> they<\/span> got<\/span> it<\/span> illegally<\/span> in<\/span> Italy<\/span>,<\/span> and<\/span> then<\/span> had<\/span> to<\/span> give<\/span> it<\/span> back<\/span>.<\/span> But<\/span> it<\/span> was<\/span> all<\/span> worth<\/span> it<\/span> –<\/span> it<\/span> made<\/span> history<\/span>!<\/span><\/p><\/blockquote>\n

While fun, gangster-speak isn’t what we’re after. Instead, we wondered, could we get GPT-3 to summarize these articles about the antiquities trade as a knowledge graph by generating the necessary code directly? Turns out, yes, yes you can.<\/p>\n

We started with this prompt:<\/p>\n

Write correctly formatted Cypher create and match queries \r\nto represent the knowledge graph of this text.\r\n<\/pre>\n
and it duly wrote,<\/p>\n
CREATE (medici:Person {name:\"Giacomo Medici\"}), \r\n(hecht:Person {name:\"Robert Hecht\"}), \r\n(krater:Artifact {name:\"Euphronios (Sarpedon) krater\"}), \r\n(romana:Gallery {name:\"Antiquaria Romana\"}), \r\n(cleveland:Museum {name:\"Cleveland Museum of Art\"}), \r\n(cooney:Person {name:\"John Cooney\"}), \r\n(italy:Country {name:\"Italy\"}), \r\n(medici)-[:STARTED_DEALING_IN_ANTIQUITIES_IN]->(italy), \r\n(medici)-[:BECAME_SUPPLIER_TO]->(hecht), \r\n(medici)-[:OPENED]->(romana), \r\n(medici)-[:BOUGHT]->(krater), \r\n<\/pre>\n
<snip><\/p>\n
\"\"
Code above dropped directly into Neo4j<\/p><\/div>\n
<\/p>\n
This code works! The problem though is that if you run a second article through the OpenAI API that mentions some of the same individuals, objects, places, or organizations, there’s no memory<\/i> of the previous time. So you’ll get different variable names being created for the same figure, and so multiple nodes in your graph that are actually the same person or thing. We think we can get around this problem though by carefully framing the prompt with a few limited examples (the longer the prompt text, the shorter the response, so that’s also a concern).<\/p>\n
This is what we’re working with at present, which seems to handle things with only a few errors that can be cleared up manually afterwards:<\/p>\n
Write a correctly formatted Cypher create query \r\nto represent first key individuals, organizations, and objects, \r\nthen the appropriate relationships between them. \r\nCheck to see if a node exists before creating it; \r\nALWAYS derive the variable name from the first initial plus the name. \r\nHere is an example of the desired output: \r\nMERGE (gfb:Individual {name: 'Gianfranco Becchina'})\r\nMERGE (gkk:Object {name: 'Getty Kouros'})\r\nCREATE (gfb)-[:ACQUIRED]->(gkk)\r\n<\/pre>\n
The screenshot below is the result after running four articles mentioning the figure of Giacomo Medici (from the Trafficking Culture Project Encyclopedia<\/a>) through our prompts; total elapsed time to get the Cypher statements: about 3 minutes.<\/p>\n
We’ve still got some kinks to figure out, but it’s exciting to see this (mostly) working. Once the graph is built, we’ll be able to query it too using natural language instead of Cypher – we’ll just ask GPT3 to translate our questions into suitable code.<\/p>\n
\"\"<\/p>\n
Things to Read
\n<\/strong><\/p>\n
https:\/\/neo4j.com\/developer-blog\/explore-chatgpt-learning-code-data-nlp-graph\/<\/a><\/p>\n
https:\/\/towardsdatascience.com\/gpt-3-for-doctor-ai-1396d1cd6fa5<\/a><\/p>\n
<\/p>\n
<\/p>\n","protected":false},"excerpt":{"rendered":"
One of the things we’re working on in our group are ways to extract structured knowledge from lots of unstructured text. In an article coming out soon in Advances in Archaeological Practice, we demonstrate some of the things that we can accomplish when we have statements about a domain (in this case, the antiquities trade) […]<\/p>\n","protected":false},"author":6,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_relevanssi_hide_post":"","_relevanssi_hide_content":"","_relevanssi_pin_for_all":"","_relevanssi_pin_keywords":"","_relevanssi_unpin_keywords":"","_relevanssi_related_keywords":"","_relevanssi_related_include_ids":"","_relevanssi_related_exclude_ids":"","_relevanssi_related_no_append":"","_relevanssi_related_not_related":"","_relevanssi_related_posts":"","_relevanssi_noindex_reason":"","_mi_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":"","_links_to":"","_links_to_target":""},"categories":[1],"tags":[58,57,59,27],"yoast_head":"\nKnowledge Graphs and GPT-3 - X-Lab<\/title>\n<meta name=\"description\" content=\"One of the things we're working on in our group are ways to extract structured knowledge from lots of unstructured text. In an article coming out soon in\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/carleton.ca\/xlab\/2023\/knowledge-graphs-and-gpt-3\/\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"shawngraham\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/carleton.ca\/xlab\/2023\/knowledge-graphs-and-gpt-3\/\",\"url\":\"https:\/\/carleton.ca\/xlab\/2023\/knowledge-graphs-and-gpt-3\/\",\"name\":\"Knowledge Graphs and GPT-3 - X-Lab\",\"isPartOf\":{\"@id\":\"https:\/\/carleton.ca\/xlab\/#website\"},\"datePublished\":\"2023-02-02T20:28:28+00:00\",\"dateModified\":\"2023-05-24T18:43:26+00:00\",\"author\":{\"@id\":\"https:\/\/carleton.ca\/xlab\/#\/schema\/person\/e8707158a71e77734ea13346b6e46feb\"},\"description\":\"One of the things we're working on in our group are ways to extract structured knowledge from lots of unstructured text. In an article coming out soon in\",\"breadcrumb\":{\"@id\":\"https:\/\/carleton.ca\/xlab\/2023\/knowledge-graphs-and-gpt-3\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/carleton.ca\/xlab\/2023\/knowledge-graphs-and-gpt-3\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/carleton.ca\/xlab\/2023\/knowledge-graphs-and-gpt-3\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/carleton.ca\/xlab\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"News\",\"item\":\"https:\/\/carleton.ca\/xlab\/category\/news\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Knowledge Graphs and GPT-3\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/carleton.ca\/xlab\/#website\",\"url\":\"https:\/\/carleton.ca\/xlab\/\",\"name\":\"X-Lab\",\"description\":\"Carleton University\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/carleton.ca\/xlab\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/carleton.ca\/xlab\/#\/schema\/person\/e8707158a71e77734ea13346b6e46feb\",\"name\":\"shawngraham\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/carleton.ca\/xlab\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/1b4be5c0f305aa12c7a3dd75ae5c731e?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/1b4be5c0f305aa12c7a3dd75ae5c731e?s=96&d=mm&r=g\",\"caption\":\"shawngraham\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Knowledge Graphs and GPT-3 - X-Lab","description":"One of the things we're working on in our group are ways to extract structured knowledge from lots of unstructured text. In an article coming out soon in","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/carleton.ca\/xlab\/2023\/knowledge-graphs-and-gpt-3\/","twitter_misc":{"Written by":"shawngraham","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/carleton.ca\/xlab\/2023\/knowledge-graphs-and-gpt-3\/","url":"https:\/\/carleton.ca\/xlab\/2023\/knowledge-graphs-and-gpt-3\/","name":"Knowledge Graphs and GPT-3 - X-Lab","isPartOf":{"@id":"https:\/\/carleton.ca\/xlab\/#website"},"datePublished":"2023-02-02T20:28:28+00:00","dateModified":"2023-05-24T18:43:26+00:00","author":{"@id":"https:\/\/carleton.ca\/xlab\/#\/schema\/person\/e8707158a71e77734ea13346b6e46feb"},"description":"One of the things we're working on in our group are ways to extract structured knowledge from lots of unstructured text. In an article coming out soon in","breadcrumb":{"@id":"https:\/\/carleton.ca\/xlab\/2023\/knowledge-graphs-and-gpt-3\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/carleton.ca\/xlab\/2023\/knowledge-graphs-and-gpt-3\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/carleton.ca\/xlab\/2023\/knowledge-graphs-and-gpt-3\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/carleton.ca\/xlab\/"},{"@type":"ListItem","position":2,"name":"News","item":"https:\/\/carleton.ca\/xlab\/category\/news\/"},{"@type":"ListItem","position":3,"name":"Knowledge Graphs and GPT-3"}]},{"@type":"WebSite","@id":"https:\/\/carleton.ca\/xlab\/#website","url":"https:\/\/carleton.ca\/xlab\/","name":"X-Lab","description":"Carleton University","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/carleton.ca\/xlab\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/carleton.ca\/xlab\/#\/schema\/person\/e8707158a71e77734ea13346b6e46feb","name":"shawngraham","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/carleton.ca\/xlab\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/1b4be5c0f305aa12c7a3dd75ae5c731e?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/1b4be5c0f305aa12c7a3dd75ae5c731e?s=96&d=mm&r=g","caption":"shawngraham"}}]}},"acf":{"Post Thumbnail Icon":"","Post Thumbnail":false},"_links":{"self":[{"href":"https:\/\/carleton.ca\/xlab\/wp-json\/wp\/v2\/posts\/86"}],"collection":[{"href":"https:\/\/carleton.ca\/xlab\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/carleton.ca\/xlab\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/carleton.ca\/xlab\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/carleton.ca\/xlab\/wp-json\/wp\/v2\/comments?post=86"}],"version-history":[{"count":3,"href":"https:\/\/carleton.ca\/xlab\/wp-json\/wp\/v2\/posts\/86\/revisions"}],"predecessor-version":[{"id":111,"href":"https:\/\/carleton.ca\/xlab\/wp-json\/wp\/v2\/posts\/86\/revisions\/111"}],"wp:attachment":[{"href":"https:\/\/carleton.ca\/xlab\/wp-json\/wp\/v2\/media?parent=86"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/carleton.ca\/xlab\/wp-json\/wp\/v2\/categories?post=86"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/carleton.ca\/xlab\/wp-json\/wp\/v2\/tags?post=86"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}