Similiar experience here, with most of my frustration with the unsuable formating of the returned JSON.
Its was difficult to know if it was a problem with something I just did to my prompt or not, so it slowed my progress.
The docs for gpt-3.5-turbo say its the 'best model for many non-chat use cases' at a reduce cost. I played with it as well, for me it seems to perform much worse than text-davinci-003. Again, not sure if that's my prompt or not.
Sending the entire prompt each time did appear to be the only option. I also experience it making up rooms, so I'm giving it rooms now. Which allows you to say turn off the X in the Family Room or all of the lights in the X room.
I put in, took, out then put some verbiage about who I am so it would know if I said turn off 'my whatever' it could from the device name know what I was talking about in any room. Then...I thought about, what about my partners stuff. Then....what about....this quirk in my home. I should explain that.
So with a growing prompt, I looked into pricing. Currently my prompt is 1,977 tokens or 6714 characters.
(https://platform.openai.com/tokenizer)
The gpt-3.5-turbo model is priced at $0.002 per 1k tokens. (https://openai.com/blog/introducing-cha ... isper-apis)
Someone double check me on ALL of this, thats $0.004 per request. So if I make 50 requests per day over 30 days, that's 1500 requests @ $0.004 per request which is $6 USD/mo rounding up. lol. Interestingly this is a subject ChatGTP is not so well versed in. However, ChatGTP did explain the ChatGTP Plus users do not get any free tokens and pricing for the API is totally separate.
Do you think you can work thru the unreadable JSONs or are just accepting you'll have to toss some responses?