Looking for ideas on chunking reservation related conversations and context aware RAG

Hey guys, i had discussed this briefly a few meetings back about the problem I am trying to solve. I am stuck on how to best create the vector store, as simply chunking all messages and using some kind of Sentence splitting does not work well

I manage a property portfolio on platforms like Airbnb, handling customer support through the entire guest journey (pre-booking to post-stay). I'm building a RAG system to help automate responses to guest inquiries.

Here are the questions I have - some context of business below

## Technical Questions

1. **Vector Database Strategy**

- How to structure embeddings for different information types?

- Chunking Strategy Challenges:

- Single message chunks: Lose conversation context

- Multi-message chunks: How many messages maintain coherence?

- Entire conversation chunks: May be too broad for specific queries

- How to preserve booking context (guest state, property details) within chunks?

- Should property-specific and global information be in separate vector spaces?

- How to handle property hierarchies in vector search?

2. **Temporal Relevance**

- How to weight conversation recency differently based on query type?

- How to combine current property documents with historical conversations?

3. **Context-Aware Retrieval**

- How to incorporate guest journey state into the retrieval process?

- How to handle property relationships (e.g., similar apartments sharing info)?

- How to balance property-specific vs. global policy information?

4. **Security and Policy Compliance**

- How to ensure RAG responses respect security policies based on guest journey state?

- How to handle platform-specific rules in responses?

## Data Sources and Unique Challenges

### 1. Historical Conversations (around 10,000 reservations over 7 years , each having 10-40 messages during the client journey)

- Stored in PostgreSQL

- Time relevance varies by query type:

```

Example A: "What's the WiFi password?"

→ Recent conversations only relevant (passwords change)

Example B: "Is there a park nearby?"

→ Older conversations still valuable

```

### 2. Property Information

- Detailed & up to date property descriptions (amenities, location, rules, what is around, any specific rules for the property)

- Properties have hierarchical relationships:

```

Building

├── Apartment A

├── Apartment B (identical layout/appliances to A)

└── Apartment C (different layout but same building amenities)

```

Question: "I am having issue with turning the cooking hob on for Property X".

(X is in a building with several other properties)

→ RAG should be done on Property X but also on other properties in the same building

```

### 3. Guest Journey States

Each conversation happens in one of these states:

```

BEFORE_BOOKING → AFTER_BOOKING → ON_DAY_OF_ARRIVAL → DURING_STAY → AFTER_STAY

```

The same question might need different answers based on the state:

```

Q: "What's the lockbox code?"

- BEFORE_BOOKING: Security risk, don't share

- AFTER_BOOKING: Still don't share

- ON_DAY_OF_ARRIVAL: Share code + flag to human to check why it wasn't sent earlier

```

### 4. Other info about business and sales channels

- Global policies (checkout times, cancellation rules)

- Platform-specific rules (Airbnb vs. other platforms). E.g. Airbnb has a rule of not asking for photo ID before booking is confirmed

## EXAMPLE RESERVATION DETAILS & CONVERSATION

- **Guest Name:** John Doe

- **Flat:** 23 Kensington Gardens, London *[Flat is in building X with other properties Flat 23, Flat 24, Flat 25]*

- ** Sales Channel:** Airbnb

- **Enquiry Created:** 21 May 2024, 9:47 pm

- **Stay Duration:** 09 Nov 2024 - 16 Nov 2024 (7 nights)

- **Number of Tenants staying:** 3

### Chronologically ordered messages between Guest and Host

##### Note: Notation:

- (format: Date Time | Platform | Sender : Message Text)

- Sender is Guest if the message is from Guest, Host if the message is from us (we are the host)

- Ordered Oldest to newest

### All messages

```

PRE BOOKING

```

- 21 May 2024, 9:47 pm | Platform | Guest: "Good evening, sir. Is the flat available from the 9th to the 16th of November for two guests? Kind regards"

- 21 May 2024, 9:49 pm | Platform | Host: "Hi John, Thanks for your interest in booking our property. 😊 We have noticed that you don't have many reviews on Airbnb platform, we are somewhat wary due to some some unfortunate circumstance. Please give us a brief description of your stay, so we can check if we can fully accommodate your needs too. Cheers to a great stay!"

> *This is a standard question that we ask based on some criterion about the number of reviews the guest has on the Airbnb platform. This is dealt with in **SALES_CHANNEL_DOCUMENTS** -*

- 21 May 2024, 10:03 pm | Platform | Guest: "It's a simple trip to visit London, nothing fancy. We prefer to book a flat than a hotel to have more personal space and cook our meals instead of paying for each meal. I don't have any reviews yet because I created my account yesterday because I usually use other sites to book my stays. We really liked your flat's design - it's lovely! - and we think its price is pretty fair. (We're from Italy and we're 25 and 28). Thank you for your time, Hope to hear from you soon, John"

```

AFTER BOOKING

```

- 24 May 2024, 8:02 pm | Platform | Host: "Hi John, Thanks for booking with us! We're excited to Host: - Photo of your ID - Name and age of the other guest/s - UK phone number and email address - Time of your arrival Please also share with us the brief reason of stay 😊 Best regards"

- 29 May 2024, 2:32 pm | Platform | Guest: "Hi , sorry for the late reply, it's been hectic! Do you also need the other guest's ID? I'm going to send you everything. My name is John Doe , age 25, and the other is Jane XX , age 28 for when we'll be there (now they're 27). We don't have a British phone number, but you can text me on whatsapp at +XXXX and my email is YYYY. We'll arrive around noon. Thank you, kind regards, John"

- 29 May 2024, 2:36 pm | Platform | Guest: "AB Message - No Description" **(Sends photo ID)**

- 29 May 2024, 3:12 pm | Platform | Host: "please confirm if you are fine with our house rules"

- 30 May 2024, 10:53 am | Platform | Guest: "We're fine with the house's rules."

- 30 May 2024, 10:54 am | Platform | Guest: "We'll arrive for the check-in around 4pm as it's written."

- 30 May 2024, 1:45 pm | Platform | Guest: "AB Message - No Description" **(Sends photo ID)**

- 30 May 2024, 1:46 pm | Platform | Guest: "This is the other guest's passport."

- 30 May 2024, 1:48 pm | Platform | Host: "Hi John, Thanks for sending over your details. Below you will find a link with directions to reach your property and other useful info. These will help you to plan your trip easier. https://LINK **(when LLM needs to answer to this kind of "info" giving message, this link will be generated by function calling)** We will send you detailed instructions about the flat and how to access it on the day of your check in. Please bear in mind that our check-in time starts at 4pm on the day . Checkout is by 10am on the checkout date. Best regards,"

- 22 Jul 2024, 5:22 pm | Platform | Guest: "Hi Jason, sorry to bother you but Ryanair changed our flight times. We would arrive at London Stansted Airport around 9pm. Until what time can we do the check-in? Thank you, John"

- 22 Jul 2024, 5:25 pm | Platform | Host: "Hi John, No worries, you can check in anytime 😊"

- 22 Jul 2024, 5:26 pm | Platform | Guest: "Ok,thank you so much!"

- 22 Oct 2024, 8:54 pm | Platform | Guest: "Hello Jason, I wanted to inform you that we paid the whole amount for our stay in your flat. Thank you so much, John"

```

ON DAY OF ARRIVAL

```

- 29 Oct 2024, 11:37 am | Platform | Guest: "Hey there Jason. We'll arrive around 11pm (if everything goes well with the transport). Do we have to do anything else? Will you give us the check in instructions the day of our arrival? Thank you so much, have a lovely day, John"

- 31 Oct 2024, 11:05 pm | Platform | Guest: "Jason, is everything ok? Are there any problems?"

- 08 Nov 2024, 9:28 pm | Platform | Host: "Hi John, Your arrival is due soon. Please find below the link to the Johnf check-in information for your property. We have included photos and videos on how to find and enter into the property together with the instructions for the appliances, WIFI, local shops etc. Please confirm as soon as you have seen these instructions, I will sleep better knowing that you will enter the property smoothly. KEYS COLLECTION Please pick up the keys before going to the property. Your key pick up code, at the store, is 804446. This key collection service is by Keynest a trusted partner. DETAILED CHECK-IN https:///XXXX Have a great stay."

- 09 Nov 2024, 1:33 pm | Platform | Guest: "Thank you so much, we read everything and soon we'll head to the airport. Thank you again, John"

- 09 Nov 2024, 10:51 pm | Platform | Guest: "We arrived at the flat👍🏻"

- 10 Nov 2024, 8:10 am | Platform | Guest: "Good morning, Jason. We tried to login into the wifi connection but it requires some kind of authorisation. How can we solve this problem?"

- 11 Nov 2024, 8:28 am | Platform | Host: "Hi John, Thannks for letting us know. Can you give us a screenshot of it? So we can check? Best regards"

- 11 Nov 2024, 10:33 am | Platform | Host: "Hello John, We are expecting a delivery of the router from Royal mail today, please accept the delivery between 10:12am - 2:12pm today www.abc.com/XXXXXX Kindly confirm as soon as the new router is received Thank you."

- 11 Nov 2024, 10:34 am | WhatsApp Direct | Host: "Hello John, We are expecting a delivery of the router from Royal mail today, please accept the delivery between 10:12am - 2:12pm today www.abc.com/XXXXXX Kindly confirm as soon as the new router is received Thank you."

- 11 Nov 2024, 5:52 pm | Whatsapp | Guest: "Sorry, I didn't see the message, we just got at the flat."

- 11 Nov 2024, 5:55 pm | Whatsapp | Host: "No problem. Thanks"

- 11 Nov 2024, 6:26 pm | Whatsapp | Guest: "Is there something we can do?"

- 11 Nov 2024, 6:38 pm | Whatsapp | Host: "Hi John, All good for now. Thanks for asking"

- 11 Nov 2024, 7:37 pm | Whatsapp | Guest: "What do we do with the WiFi issue?"

- 11 Nov 2024, 7:57 pm | Whatsapp | Host: "Hi John, Our cleaner will pick up the router tomorrow and will install it, issue will be resolve tomorrow, sorry for the inconvenience."

- 11 Nov 2024, 9:36 pm | Whatsapp | Guest: "Thank you so much! Have a good night

8 comments