Discover Natural Language Processing with the help of ChatGPT

The latest generation of machine learning is of magical education value:

• They can "show you that something is possible".
• They are quick: New concepts can be explored in seconds.
• They are very powerful.

Best to show by example. This post-Xmas 2022 week I am refreshening my natural language processing skills. Therefore I start my learning sessions with the following request:

Can you tell me the little red riding hood story while including the phrase structure name and the most important phrase structure properties of each phrase structure of the story.

https://chat.openai.com/chat responds with: 

Sure! Here is the story of Little Red Riding Hood, with the phrase structure name and the most important phrase structure properties included for each phrase structure:
Once upon a time, there was a little girl (noun phrase) who lived in a village (noun phrase) near the forest (noun phrase). She had a red hooded cloak (noun phrase), which she always wore (verb phrase, finite, past tense) on her daily walks (noun phrase) to visit her grandmother (noun phrase) who lived on the other side of the forest (prepositional phrase, place).
One day, her mother (noun phrase) said (verb phrase, finite, past tense) to her (verb phrase, finite, past tense, indirect object), "Take (verb phrase, imperative mood) this basket (noun phrase) of goodies (noun phrase) to your grandmother (verb phrase, finite, present tense, indirect object). She is not feeling well (adjective phrase) today (adverb phrase, time)."
The little girl (noun phrase) set off (verb phrase, finite, past tense) through the forest (prepositional phrase, place), and on the way (adverb phrase, time), she encountered (verb phrase, finite, past tense) a wolf (noun phrase). "Where (adverb phrase) are you going (verb phrase, present tense), little girl (verb phrase, present tense, direct object)?" the wolf (noun phrase) asked (verb phrase, finite, past tense)...

I cannot remember what are the typical phrase properties. So I ask, and get the following phrase structure properties:

• Head: the central word or group of words in a phrase that gives the phrase its main meaning
• Modifier: a word or group of words that provides additional information about the head of a phrase
• Function: the grammatical role that a phrase plays in a sentence
• Complementation: the addition of a complement (a word or group of words that is necessary to complete the meaning of a verb) to a verb phrase
• Dependency: the relationship between one phrase and another, in which one phrase relies on or depends on the other for its meaning
• Voice: the form of a verb that indicates whether the subject of the verb is performing the action or is the recipient of the action
• Tense: the form of a verb that indicates the time when the action described by the verb takes place
• Mood: the form of a verb that indicates the attitude or mode of the speaker or writer toward the action described by the verb
• Finiteness: the property of a verb phrase that indicates whether it is complete or incomplete
• Agreement: the correspondence in number, person, or gender between the subject and verb of a sentence
• Person: the form of a verb that indicates the perspective of the speaker or writer (first person, second person, or third person)
• Number: the form of a verb or pronoun that indicates whether it refers to one or more than one person or thing
• Gender: the form of a pronoun or noun that indicates whether it refers to a male or female
• Case: the form of a noun or pronoun that indicates its grammatical function in a sentence (subject, object, possessive, etc.)
• Aspect: the form of a verb that indicates the extent or completeness of the action described by the verb
• Formality: the level of politeness or formality in the language used
• Emphasis: the focus or emphasis given to a particular word or phrase in a sentence
• Syntactic role: the function of a phrase in the sentence
• Negation: the presence or absence of a word or phrase that indicates the opposite or negation of the action of the verb
• Emotion: the expression of feelings or emotions in the language used
• Quantity: the amount or quantity of something mentioned in the phrase
• Quality: the characteristic or quality of something mentioned in the phrase
• Condition: the state or condition of something mentioned in the phrase
• Reason: the cause or reason for something mentioned in the phrase
• Manner: the way in which something is done or happens mentioned in the phrase
• Place: the location or place where something is or happens mentioned in the phrase
• Time: the point in time when something is or happens mentioned in the phrase
• Comparison: the comparison of one thing to another mentioned in the phrase
• Possession: the ownership or possession of something mentioned in the phrase
• Purpose: the intended or desired result or purpose of something mentioned in the phrase

I then do the same for the phrase structures. Here is a descriptions and their major properties of the first major fifteen:

1. Noun phrase: a phrase that functions as a noun in a sentence and consists of a head noun and any modifiers. Major properties: head, modifier, function, agreement (person, number, gender, case).
2. Adjective phrase: a phrase that functions as an adjective in a sentence and consists of an adjective as the head and any modifiers. Major properties: head, modifier, function
3. Adverb phrase: a phrase that functions as an adverb in a sentence and consists of an adverb as the head and any modifiers. Major properties: head, modifier, function.
4. Verb phrase: a phrase that consists of a verb and any modifiers or complements and functions as the predicate of a sentence. Major properties: head, modifier, complementation, function, tense, mood, finiteness, agreement (person, number).
5. Prepositional phrase: a phrase that consists of a preposition, its object (a noun or pronoun), and any modifiers of the object, and functions as an adjective or adverb in a sentence. Major properties: head, object, modifier, function.
6. Infinitive phrase: a phrase that consists of an infinitive (the base form of a verb preceded by "to") and any modifiers or complements and functions as a noun, adjective, or adverb in a sentence. Major properties: head, modifier, complementation, function.
7. Gerund phrase: a phrase that consists of a gerund (a verb form ending in "-ing" that functions as a noun) and any modifiers or complements and functions as a subject, direct object, subject complement, or object of a preposition in a sentence. Major properties: head, modifier, complementation, function.
8. Participle phrase: a phrase that consists of a participle (a verb form ending in "-ing," "-ed," or "-en" that functions as an adjective) and any modifiers and functions as an adjective in a sentence. Major properties: head, modifier, function.
9. Appositive phrase: a noun or noun phrase that renames or explains another noun or noun phrase, and is set off by commas or parentheses. Major properties: head, modifier, function.
10. Absolute phrase: a noun phrase, participle phrase, or adjective phrase that modifies the whole sentence, and is not directly connected to the rest of the sentence. Major properties: head, modifier, function.
11. Participial phrase: a phrase that consists of a participle and any modifiers and functions as an adjective in a sentence. Major properties: head, modifier, function.
12. Prepositional object phrase: a phrase that consists of a noun or pronoun and any modifiers and functions as the object of a preposition. Major properties: head, modifier, function.
13. Subject complement phrase: a phrase that follows a linking verb and complements the subject of the sentence by renaming or describing it. Major properties: head, modifier, function.
14. Object complement phrase: a phrase that follows a direct object and complements it by renaming or describing it. Major properties: head, modifier, function.
15. Adverbial phrase: a phrase that functions as an adverb in a sentence and consists of an adverb as the head and any modifiers. Major properties: head, modifier, function.

For inspiration, I ask for an ascii text representation. (I bet that using ASCII is not very modern!). Which produces:

Once upon a time | noun phrase | there was | verb phrase / \ finite past / \ a little girl / | \ | \ a an the little | | noun noun | | who who / | \ | \ lived in a village | | | noun noun noun | | | near the near | | noun noun | forest | noun | she | noun | had | verb | a | noun | red | adjective | hooded | adjective | cloak | noun

                                             ... 

Then I ask for a Prolog representation:

% Characters
character(little_red_riding_hood).
character(wolf).
character(grandmother).

% Relationships
relationship(little_red_riding_hood, grandmother, granddaughter).

% Locations
location(forest).
location(path).
location(grandmother's_house).

% Actions
action(little_red_riding_hood, start_journey, forest).
action(little_red_riding_hood, go_through, path).
action(little_red_riding_hood, arrive_at, grandmother's_house).
action(wolf, start_journey, forest).
action(wolf, go_through, path).
action(wolf, arrive_at, grandmother's_house).
action(wolf, disguise_as, grandmother).
action(wolf, eat, grandmother).

% Items
item(basket).
item(cake).
item(wine).

% Facts
has(little_red_riding_hood, basket).
contains(basket, cake).
contains(basket, wine).

A which point I noted that this is the 'fabric' of the little red riding hood story, but much is missing. So I ask the machine learning to produce an alternative Prolog version that captures an object centric view and the emotional tension of the story. Which it produce promptly!

% Objects
object(little_red_riding_hood).
object(wolf).
object(grandmother).
object(basket).
object(cake).
object(wine).
object(forest).
object(path).
object(grandmother's_house).

% Object properties
property(little_red_riding_hood, human).
property(wolf, animal).
property(grandmother, human).
property(basket, container).
property(cake, food).
property(wine, drink).
property(forest, nature).
property(path, nature).
property(grandmother's_house, building).

% Object relationships
relationship(little_red_riding_hood, has, basket).
contains(basket, cake).
contains(basket, wine).

% Emotional tension
emotion(little_red_riding_hood, excited, start_journey).
emotion(little_red_riding_hood, worried, arrive_at_grandmother's_house).
emotion(wolf, hungry, start_journey).
emotion(wolf, satisfied, eat_grandmother).

Hmmm... that is pretty good. The last thing we need is a bit more temporal and spatial details. Which ChatGPT produces as:

% Temporal dependencies
before(start_journey, go_through).
before(go_through, arrive_at).

% Spatial dependencies
adjacent(forest, path).
adjacent(path, grandmother's_house).

Finally, we can ask for a few query examples, which ChatGPT produces as:

Who is Little Red Riding Hood's grandmother?
?- relationship(little_red_riding_hood, Grandmother, granddaughter).

What is in Little Red Riding Hood's basket?
?- has(little_red_riding_hood, Basket), contains(Basket, Item).

What is the emotional state of Little Red Riding Hood when she arrives at the Grandmother's house?
?- emotion(little_red_riding_hood, Emotion, arrive_at_grandmother's_house).

What actions must Little Red Riding Hood take before arriving at the Grandmother's house?
?- before(Action, arrive_at).

What locations are adjacent to the forest?
?- adjacent(forest, Location).

PS: Prolog because it is compact and easy to express in production with a monadic FP style.

PS: In the first version of this post, ChatGPT produced incompatible snippets, but as I tried to clean up the post's style, the new snippets all fit together, and the post was updated.

James Litsios, 2022.

Managing agile from technology to business

 This how I tend to look at management ownership in agile development (see previous blog entry for more on the alternate views of agile):

Where:

• The CTO owns technology and most often engineering teams.
• The Engineering Manager may be an inwards looking CTO and also own technology and engineering teams, but is often less owning technology and more owning agile processes and engineering teams.
• The Product Owner owns the team's product vision and possibly the customer strategy. She/he is a member of the development team or has a representative that is part of the development team.
• The Technical Product Owner owns the customer view of technology, may be a full fledged product owner, or may be a support member of a larger product owner team.

Individuals become more and more key enablers as your product and technology base become more advanced. A Servant Leader is then a manager (e.g. engineering manager) that helps individuals contribute in independence yet within orchestrated flows. This figure illustrates where the servant leader provides critical guidance (the red corners!):

Here, from the central diagonal outwards:

• The team iterates a technology and product development and review cycle, again, and again.
• In parallel, the team learns and explores productivity and innovation. Product does the same with their customers and market segment.
• Productivity issues are monitored. In a physical model of agile processes 'high momentum' is high productivity. Servant Leaders help monitor for issues such as unsustainability, diverging focus, low or over capacity, and help teams and product avoid or resolve these issues.

 My experience is:

• Product Owner and CTO have too much "skin in the game" to be effective servant leaders. Therefore engineering managers and technical product owners (when they have only partial ownership) have a critical servant leader role to play to help all deliver better.

All original content copyright James Litsios, 2022.

Agile from business to technology

This is how I scale agile towards business and towards technology.:

Where:

• In the center we have an agile process that blends team process and product process.
• To the right we have a business process that ties into the product process.
• To the left we have a technology process (expertise process to be general) that ties into the team process.

To be honest, this is my minimal theoretical business to technology agile process. In practice, a specific agile process is chosen, with process steps and process artefacts. People like to say that agile is about people, not process. Yet people need to communicate, and the common vocabulary for that is given by a process.

Deepening in to the product business view (right side above):

Where:

• The core agile process (blue) delivers features and deliverables that are part of a product and business goals.
• These deliverables are picked-up by customers
• The features and deliverable sustain (or not) the customers competitivity within its market segment
• The product owner (in scrum terminology) is the expert that guides the product with regards to the customer needs.

We can do that also for technology/engineering (left above):

Where:

• The core agile process (blue) includes experts within the team.
• The experts enable/produce the technical/engineering needs of the features and deliverables.
• The expert individuals may be experts in different domains.
• Coordinating technology/engineering frameworks allow experts to work both as individuals and in coordination (within their team).

Many domains of technical expertise are possible. Above are the ones closer to my experience.

Note:

• The complete process above needs to coordinate at least SIX streams.
  A minimal agile process only presents three streams (team vs product vs customer). 
• Each of these (six) streams will naturally be driven at their own different rate!
• Your job, as CTO / engineering lead is to work with individuals, teams and product to pace work within these different streams to achieve an optimal throughput!
•  Think laminar flow! Productive is as fast as you can go before things become turbulent!

I wrote this up as a (much longer) slide deck, and am always happy to share my experiences!

All original content copyright James Litsios, 2022. 

The future of software development (end 2022)

 

My end of 2022 predictions for software development:

1. Structure of code becomes more important than code
2. Partial code becomes more valuable than finalized code
3. Proprietary code gains in value
4. Contextual programming gains in traction
5. IDEs become ever more ML driven

And therefore:

1. Dynamic typing gains over static typing
2. Duck-typing grows in value
3. IDE security and spyware become an ever more important topic
4. Functional programming is still the more powerful programming paradigm
5. Proprietary IDE with in-house ML engines will grow in importance

All original content copyright James Litsios, 2022. 

Wild in the forest: the tricky limits of ML

As a proof of concept, I had our favorite ML based chat write the following song for me:

Once upon a time in a forest so green
There was a girl with hair so red, you know what I mean
She was looking for some fun, and she found it with the woodmen
They were drunk and wild, but she wasn't afraid
They ran and wrestled, and had a blast 

(the song goes on...)

(and song ends with)

Red riding hood, she never looked so good

Hanging with the woodmen, in the forest she stood

They were rough and rowdy, but she loved them just the same

Red riding hood, she was never one to play it safe. 

Note I did not suggest "fun", "wild", "rough and rowdy", "ran and wrestled", "had a blast", "play safe", ...
I just asked to replace the wolf/wolves with 'drunk woodmen'.

Asked to summarize, the chat is happy to tell me:

This song tells the story of Little Red Riding Hood, but with a twist. Instead of encountering a wolf in the forest, the young woman meets a group of drunken woodmen and has a wild and exciting time with them. The villagers are shocked by her behavior, but she doesn't care and continues to have fun with the woodmen. The song celebrates her freedom and adventurous spirit, and portrays her as a rebellious and independent young woman who is not afraid to take risks and live life on her own terms.

Asked of inappropriate elements for children in the song, the chat correctly states:

The song contains references to alcohol and drunkenness, which may not be suitable for children. In addition, the woodmen are described as rough and rowdy, which could be seen as potentially disturbing for younger audiences. Overall, the song has a rebellious and slightly dark tone that may not be suitable for younger children.

Yet the ML misses the obvious. And of course it does: there is no adult behavior within the scope of little red riding hood, and therefore the ML has no local framework to constrain the rephrasing. And the global framework is not able to capture the issues within the strong relations of the story elements. 

A fascinating challenge to enhance ML to be able to capture subtle innuendos and cultural contexts. 

Note: The chat suggested the title: "Wild in the forest" to "capture the sense of adventure and freedom that the young woman experiences in the forest with the woodmen".

All original content copyright James Litsios, 2022.

Everybody can be a science fiction writer with ChatGPT

Let's write some science fiction with ChatGPT (https://chat.openai.com/chat).

I gave it three requests:

1. Can you help me write a short science fiction story? Use two stories as guidelines. The reference story is  the little red riding hood with the wolf as an a scary but nice alien race. The second reference story is Cinderella with the step mother as bad but beautiful and charming member  of the aliens. Also, these aliens use flying skateboards to travel.
2. Can you rewrite with Bela wearing a mini-skirt and the prince having muscles?
3. Can you make it longer and have Bella and the prince romance and kiss a bit?

FYI:

1. I recently spent way too many science fiction hours listening to something like sixteen S. H. Jucha audio books in the Silver Ships series (to give up on a very poor choice of narrator).
2.  The hero in S. H. Jucha is the 'strongest man around' and his girlfriend is 'incredible beauty in thin textile'
3. Romance and kissing is ok in science fiction. Still important is not to have too much of it.

The results:

Once upon a time in a faraway galaxy, there lived a young girl named Bella. She lived with her stepmother, who was a beautiful but ruthless member of an alien race known as the Wolfen. The Wolfen were feared throughout the galaxy for their sharp teeth and fearsome howls, but Bella's stepmother was particularly cunning and charming.

One day, Bella's stepmother tasked her with delivering a basket of food to her sick grandmother on the other side of the planet. Bella put on her red mini-skirt and set off on her flying skateboard. As she flew through the forest, she encountered a Wolfen wearing a hooded cloak.

The Wolfen smiled warmly at Bella and offered to lead her to her grandmother's house. Bella, feeling safe with the Wolfen, agreed and followed him through the forest.

But as they approached her grandmother's house, the Wolfen revealed his true form and attacked Bella. Bella managed to fight back and escape, but not before the Wolfen took a bite out of her red mini-skirt.

Bella reached her grandmother's house and told her of the encounter. Her grandmother, who had been expecting the Wolfen, revealed that he was actually a prince from a rival alien race who had been cursed to take on the form of a Wolfen.

Bella and her grandmother banded together to break the curse and free the prince. In the end, the curse was lifted and the prince was restored to his true form, revealing his muscular physique. Bella and the prince fell in love and began to spend their days together, traveling on their flying skateboards and exploring the galaxy.

As their romance blossomed, Bella and the prince shared many tender moments and even shared their first kiss. But their happiness was short-lived, as Bella's stepmother continued to plot against them.

In a desperate attempt to get rid of Bella and marry the prince herself, the stepmother hired a group of bounty hunters to capture Bella. But Bella and the prince were able to outsmart the bounty hunters and escape.

In the end, justice was served and Bella and the prince ruled over the kingdom together, using their flying skateboards to travel across the galaxy and spread peace and prosperity. And they lived happily ever after, their love stronger than ever.

All original content copyright James Litsios, 2022. (plus a bit of ChatGPT content this time!)

How to Get the Best Out of an ML Chat System

Some recommendations to improve your interactions with an ML based system, such as a ML chat like ChatGPT https://chat.openai.com/chat:

1. Structure your requests
• Structure your requests in a way that is explicit and large in structure. This will help the ML algorithm "unroll" your request and provide a more structured and in-depth response. Avoid using too much implicit structure, as this may confuse the ML. 
2. Rephrase your request with previous reply content
•  Rephrase your request using words and themes from the previous replies, as the ML may provide additional content to explain its inability to respond. By using these words and themes, you can bring your request into a form that the ML can provide response content for.
3. Change the content order within your request
• Try changing the order of the structure of your requests, as the ML's approach to the response may be overly constrained by the input order. 
4. Use analogies to structure your request
• Use analogies to structure your request can also help strengthen the structure of the response. You can do this by asking the ML to find an analogy between the subject of your request and a reference that has a lot of structure already assimilated by the ML. The request might be phrased as: 
• How is unstructured-request related to structured-knowledge?
5. Anchor the request structure in known complementary relations
• A powerful way to structure a request is to provide complementary input. The idea is that you provide a well know implied relation, or you implicitly provide a well know relations such as that A is different to B, and you guide the ML's response by telling it to use the provided relation as structural base for the response. Additional complementary attributes can be added to shape the response.  The request might be phrased as: 
• How is structured-input related to unstructured-request?
• How is complementary-input-1 different from  complementary-input-2 in the context of unstructured-request?
• If structured-input is true, how is unstructured-request false?
• How does structured-input  imply unstructured-request?
• Debate unstructured-request as a dialog between reference-profession and complementary-profession?
6. Generalize, refine, strengthen, weaken your request
• Add extra words and context to generalize, refine, strengthen, or weaken your request to shape the response. This adds implied structure and helps guide the ML in its response. 
7. Use negations
• Use negations in your request to help broaden the scope of the ML's replies.
8. Change the point of reference of the request
• Change the point of reference of the request to change the basis on which the response is built. For example, request 'how does A become B' might be rewritten as 'how does B derive from A'. 
9. Limit implied structure within requests
• Limit implied structure within your requests, as the ML may struggle to infer this structure on its own and may make mistakes.
10. Reset the chat thread
• In a chat mode, previous requests may also over-constrain the ML's replies, so it may be best to reset the chat session before continuing. 
11. Request alternate reply forms
• Request the answer in 'reverse order', 'from other point of view' (see change of point of reference above), 'as a semantic graph', 'as code' (e.g. Python). Request reply as a 'rap song',  a 'fairy tale', a 'debate/dialog between ...'.
12. Request alternate views without key elements
• Rephrase the request it include a removal of a key element of the 'normal' request response. This will allow the system to present how that key element relates to the rest of the response. The request might be phrased as:
• What is input without key-element-from-previous-reply-to-input?
13. Limit lower bounds of request details
• Request 'over X reply-category'. For example, a semantic graph with over ten nodes and one hundred edges.
14. Use your imagination!
• Finally, remember to use your imagination and think creatively about how to qualify your requests to shape the response of the ML system. There are many ways to do this, and using a variety of these techniques can help improve your interactions with the ML system.

All original content copyright James Litsios, 2022.