Sure the button would improve unwarranted replies from what they were before but I think it could be done better. Here are a couple quick mock ups that I think would do a better job than just a button. Some of these might not be technical feasible and require a lot more work than it's worth but I'm putting them out there anyway.
Idea 1 - New Button, Update Reply
So for this, I just made the reply button a text box by default, instead of clicking reply, most users are going to see the bottom text box and think hey I'm used to using that form chat, and then they click into it to use instead.
I've also removed the New conversation Textbox and made the button. These two things will have more impact than just the button alone.
*** New conversation button also needs to take user view to bottom of Conversation history, so that they can see their new conversation is going to go at the bottom. Currently it stays where you are in relation to what you are reading, sometimes causing reply buttons to line up with the New conversation textbox which causes users to use the wrong box, the new button should solve this use case thou.
Idea 2 - Updated Reply with Toolbar
Second Idea, Add a Toolbar similar to what we'll have on the files tab with SharePoint UI. With New Conversation Button, Meet Now, Messaging Extension and Even Follow. You could even add a Drop down on it for "Channel Roster" (People who favorite) the channel.
When clicking New conversation, cursor jumps To bottom where the conversation would go inline with the other Threads, and you are editing in place what the thread looks like instead of working from a textbox on the bottom.