Home Search AA
Number of visits:
1535
 
VBP: SUPPORT FOR VOICE-BASED PROGRAMMING

Description

The main purpose of VOCODER is to allow its users to fulfill common programming tasks using their hands as little as possible. Indeed, the only user actions that VOCODER requires are pressing some specific keybindings, speaking out the voice command, and occasionally navigating through the code in order to reposition the cursor.

Moreover VOCODER is supposed to provide a set of instructions whose complexity sits somewhere in between word-by-word input and high-level code snippets, so as to keep the user involved in the programming task without slowing it down too much. In this scenario, we decided that most of our coding commands would have been whole statements or expressions.

The implementaion of VOCODER is basically an extesion running on VScode which is one of the most used IDE in coding development.

 

The whole project had overall no specific restrictions, other than working reasonably well, employing a free software license, and using non-commercial tools or services.

 

Features:

  • The possibility to select among camel and snake case way to represent identifiers.

  • The possibility to select between dark and light themes.

  • Push-to-talk functionality to manage the input of a command.

  • The possibility to discard an input recording.

  • The possibility to create coding block providing a voice input, especially:

    • Assignment, insertion of expressions, if and if-else statements, addition of comments, creation and call of functions, while and for loops, deletion of piece of code, multiple undo and redo.

 

The application:

Opening a python file, the extension will be loaded and he user will be able to see a microphone button that can be pressed to record and a camel icon to represent that camel case is currently set. A logging output channel is automatically opened and will help the user understand what is happening and if any errors have occurred (e.g. their command was not correctly understood).

 

 

Once the microphone button (or the corresponding keybinding for push to talk) is pressed, the microphone will appear as disable. It is now possible for the user to discard the current recording by clicking on the trash button (or the corresponding keybinding)

 

 

If the command is successfully recognised and processed, the user will see wanted action performed on the editor. It is possible for the user to see what has been understood by the model in the logging channel. In this example, we show the input of an empty if statement, where the missing condition and body are filled with placeholders. 

 

 

If the trash icon gets clicked when recording, the user will be notified, both in a notification and in the log channel.

 

 

Toggling the camel button will switch from camel case to snake case, updating the icon. It is possible to go back to camel case by clicking again on the snake icon.

 

 

Technologies:

The implementation of VOCODER required the following technologies:

Frontend: Backend: Voice Recognition:
• Node.js (typescrpt) • Python • Wit.ai
• VS code API    

 

 

 

 

Repository of the project:

Repository Link


PROJECT TEAM MEMBERS

Alessandro Passoni

Project role(s): Scrum Master

Personal info: 23 years old guy attending master level in computer science engineering at Politecnico of Milan

Skills and interests: Proactive and hard-worker interested in Software Engineering, Web development

Hobbies: Basketball, Food, Motorsport, Technology, Photo and Video  editing

E-mail: alessandro3.passoni@mail.polimi.it

Other contacts:

                         Telegram: @AlePassoni

                         Facebook: Alessandro Passoni

                         Instagram: ale_passoni.24

Francesco Amorosini

Project role(s): Product Owner

Personal info: Second-year MSc student at Politecnico di Milano, currently working part-time as AWS solution architect.

Skills and interests: Zealous and hard-worker, interested in pretty much anything that involves cloud computing.

Hobbies: Soccer, playing the guitar, videogames.

E-mail: francesco.amorosini@mail.polimi.it

Other contacts: Facebook, InstagramGitLinkedin

Alberto Cabellos Jimenez

Project role(s): Developer

Personal info: Erasmus student from Spain, studying computer science.

Skills and interests: Hard-worker, interested in programming

Hobbies: Soccer, cars and hanging out with friends

E-mail: acz20002@student.mdh.se

Other contacts: Instagram: alber1311

Jaime Pascual

Project role(s): Developer

Personal info: I'm 21 years old, I study computer science in Madrid but now I'm an Erasmus student in MDH.

Skills and interests: Commited person, hard-worker and interested in all related with computer

Hobbies: Music, tennis, videogames

E-mail: jps20002@student.mdh.se

Other contacts: Instagram: pascuu7, Facebook: Jaime Pascual 

 

Serena Allkja

Project role(s): Developer

Personal info: I'm a student at PoliMi, currently attending the second year of my Master Degree in Computer Science and Engineering.

Skills and interests: Very tidy and organised person, hard-worker.

Hobbies: drawing, colouring, petting cats.

E-mail: serena.allkja@mail.polimi.it

Other contacts:

         Telegram: @serenavxv

         Instagram: serenavxv

Hilal Taha

Project role(s): Developer

Personal info: 23 Years old student from Syria, I study a double degree software engineering program where my first year was in Italy and my Second in Sweden.

Skills and interests: very good at programming and interested in technology stuff.

Hobbies: Basketball, Video Games

E-mail: hta20001@student.mdh.se

Other contacts:

Facebook: https://www.facebook.com/HilaloSoft/

Instagram: https://www.instagram.com/hilalotaha/

 

 


PROJECT SUPERVISORS

Lorenzo Addazi (MDH)

E-mail: lorenzo.addazi@mdh.se

 

Samuele Giussani (POLIMI)

E-mail: samuele.giussani@mail.polimi.it

 


PROJECT CUSTOMERS