CORE A.I. VOICE ASSISTANT (Android Version)

CHECK OUT THE WINDOWS VERSION AS WELL: HOME/LAB VOICE CONTROLLED ASSISTANT

The program is written in android studio

*WARNING* Before continuing you should know that this instructable won't learn you how to program in android studio, it will help you to develop your own voice controlled android application only if you already have some experience on programming!

Download and install Android Studio

After creating a new blank application you need to setup the libraries that we are gonna use on this project. "A library is a collection of non-volatile resources used by computer programs, often for software development. These may include configuration data, documentation, help data, message templates, pre-written code and subroutines, classes, values or typespecifications." ~Wikipedia

import android.content.ActivityNotFoundException;
import android.content.Intent;
import android.media.MediaPlayer;
import android.os.Bundle;
import android.speech.RecognizerIntent;
import android.speech.tts.TextToSpeech;
import android.support.v7.app.AppCompatActivity;
import android.view.KeyEvent;
import android.view.View;
import android.widget.Button;
import android.widget.TextView;
import android.widget.Toast;
import java.io.File;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Calendar;
import java.util.Locale;
import java.util.regex.Pattern;

After the libraries we should set the variables up! "A variable or scalar is a storage location (identified by a memory address) paired with an associated symbolic name (an identifier), which contains some known or unknown quantity of information referred to as a value."~Wikipedia

private Button openMic;
private TextView showVoiceText;
private final int REQ_CODE_SPEECH_OUTPUT = 143;
String path = "/storage/sdcard1/Music/"; //Change the path to whatever path your music is located
String[] fileArray;
public int songNumber = 0;
MediaPlayer mPlayer = new MediaPlayer();
public boolean isPlaying = false;
public boolean isPaused = false;
int result;
TextToSpeech t1;

Let's take a look on the picture... As you can see void onCreate() is the very first part of the code that is getting executed!

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_home); 
    showVoiceText = (TextView) findViewById(R.id.showVoiceOutput);
       /*Use the following command:
showVoiceText.setVisibility(View.INVISIBLE);
to make the text box invisible*/
    musicScan();
    openMic = (Button) findViewById(R.id.button);  
    openMic.setOnClickListener(new View.OnClickListener() {
        @Override
        public void onClick(View view) {
            btnToOpenMic();
        }
    });   
t1 = new TextToSpeech(getApplicationContext(), new android.speech.tts.TextToSpeech.OnInitListener(){
        @Override
        public void onInit(int status){
            if(status == TextToSpeech.SUCCESS){
                result = t1.setLanguage(Locale.getDefault());
            }else{
                Toast.makeText(getApplicationContext(), "Feature not supported", Toast.LENGTH_SHORT).show();
            }
        }
    });}

On the previous step inside the void onCreate() we set up a void called musicScan. This void is gonna scan every file inside the "path" variable that we set on the "Variable" step.

private void musicScan(){
    File f = new File(path);
    File file[] = f.listFiles();
    fileArray = new String[file.length];
    for (int i = 0; i < file.length; ++i){
        fileArray[i] = file[i].getName();
    }
}

*COMMENT*: IF YOU HAVE A NON MUSIC FILE IN THE PATH FOLDER THIS WOULD CAUSE THE APPLICATION TO CRASH THIS COULD BE EASILY SOLVED BY CHECKING WHETHER THE FILE ENDS WITH ".mp3" BEFORE ADDING IT IN THE ARRAY!

We want to open the google's voice services so that we can have access to it's enormous data base of words that google provides!

private void btnToOpenMic() {speechStandby();}
void speechStandby (){
    if(isPlaying){
        mPlayer.pause();
        isPaused = true;
    }
    Intent intent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);
    intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL, RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);</p><p>    intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE, Locale.getDefault());
    intent.putExtra(RecognizerIntent.EXTRA_PROMPT, "Speak Now...");
    try {
        startActivityForResult(intent, REQ_CODE_SPEECH_OUTPUT);
    } catch (ActivityNotFoundException tim) {    }
}

In this part of the code we are making the application more interactive, by giving it the ability to speak

public void toSpeak(String toSpeak){    
    Toast.makeText(getApplicationContext(), toSpeak, Toast.LENGTH_SHORT).show();
    int speak = t1.speak(toSpeak, TextToSpeech.QUEUE_FLUSH, null);
}

This void is pretty complicated... "Starting another activity doesn't have to be one-way. You can also start another activity and receive a result back. To receive a result, call startActivityForResult() (instead of startActivity()).

For example, your app can start a camera app and receive the captured photo as a result. Or, you might start the People app in order for the user to select a contact and you'll receive the contact details as a result. Of course, the activity that responds must be designed to return a result. When it does, it sends the result as another Intent object. Your activity receives it in the onActivityResult() callback." ~developer.android.com

The text "Core A.I." is getting replaced with the voice command, so this voice is checking what the new text is and executes a piece of code based on what we said. *You can make the text box invisible if you want to however having it visible is an easier way to debug the application*

@Override
protected void onActivityResult(int requestCode, int resultCode, Intent data){
    super.onActivityResult(requestCode, resultCode, data);
    restartMusic();
    switch (requestCode){
        case REQ_CODE_SPEECH_OUTPUT:{
            if (resultCode == RESULT_OK && null != data){
                ArrayList<string> voiceInText = data.getStringArrayListExtra(RecognizerIntent.EXTRA_RESULTS);
                showVoiceText.setText(voiceInText.get(0));
                String command = showVoiceText.getText().toString();
                if(command.equals("exit")) {
                    finish();
                    System.exit(0);
                }else if(command.equals("music")){
                    if(!isPlaying) {
                        mPlayer.reset();
                        playMusic(path + fileArray[songNumber]);
                        showVoiceText.setText("Now playing: " + fileArray[songNumber].replace(".mp3", ""));
                    }
                }else if(command.equals("stop music")){
                    if(isPlaying) {
                        stopMusic();
                    }
                }else if(command.equals("next track") || (command.equals("next song"))){
                    if(isPlaying) {
                        songNumber = songNumber + 1;
                        if(songNumber > fileArray.length - 1){
                            songNumber = 0;
                        }
                        mPlayer.reset();
                        isPlaying = false;
                        playMusic(path + fileArray[songNumber]);
                        showVoiceText.setText("Now playing: " + fileArray[songNumber].replace(".mp3", ""));
                        }               
 		}else if(command.equals("previous track") || (command.equals("previous song"))){
                    if(isPlaying) {
                        songNumber = songNumber - 1;
                        if(songNumber < 0){
                            songNumber = fileArray.length - 1;
                        }
                        mPlayer.reset();
                        isPlaying = false;
                        playMusic(path + fileArray[songNumber]);
                        showVoiceText.setText("Now playing: " + fileArray[songNumber].replace(".mp3", ""));
                    }                
		}else if(command.equals("music list")){
                    showVoiceText.setText(Arrays.toString(fileArray) + " ");
                }else if(command.equals("what time is it")){
                    time();
                }else if(command.equals("what day is today")){
                    date();
                }else if(command.toLowerCase().contains("play")){
                    String inputSong = (command.substring(command.lastIndexOf("play") + 5));
                    String inputSongLowerCase = inputSong.toLowerCase();
                    searchSong(inputSongLowerCase);
                }
            }
        }break;
    }
}

These voids are getting the time and the date whenever they are call from the void onActivityResult()!

public void time(){    
    Calendar calendar = Calendar.getInstance();
    SimpleDateFormat mdformat = new SimpleDateFormat("HH:mm");
    String strTime = "The time is : " + mdformat.format(calendar.getTime());
    toSpeak(strTime);
}public void date(){
    Calendar calendar = Calendar.getInstance();
    SimpleDateFormat mdformat = new SimpleDateFormat("dd-MMM-yyyy");
    String strDate = "Today is: " + mdformat.format(calendar.getTime());
    toSpeak(strDate);
}

Once the program gets the voice command "PLAY" it will expect from you to tell it the song that you want it to play. Before playing it should search if the specific song actually exists on the song array that we set up previously.

public void searchSong(String songToSearch){
    try {
        Toast.makeText(getApplicationContext(), songToSearch, Toast.LENGTH_SHORT).show();
        for (int i = 0; i < fileArray.length; i++) {
            String value = fileArray[i].toLowerCase().replace(".mp3", "");
            if (Pattern.compile(Pattern.quote(songToSearch), Pattern.CASE_INSENSITIVE).matcher(value).find()) {
                songNumber = i;
                isPlaying = false;
                mPlayer.reset();
                playMusic(path + fileArray[songNumber]);
                showVoiceText.setText("Now playing: " + fileArray[songNumber].replace(".mp3", ""));
                break;
            }
        }
    } catch (IllegalArgumentException e) {
        e.printStackTrace();
    } catch (Exception e) {
        System.out.println("Exception of type : " + e.toString());
        e.printStackTrace();
    }
}

Now last but not least, we need the void that will play the song that we asked for!

private void playMusic(String song){
    if(!isPlaying) {
        try {
           mPlayer.setOnCompletionListener(new MediaPlayer.OnCompletionListener() {
               @Override
                public void onCompletion(MediaPlayer mPlayer) {
                    isPlaying = false;
                    mPlayer.reset();
                    if(songNumber > fileArray.length - 1){
                        songNumber = 0;
                    }else {
                        songNumber = songNumber + 1;
                    }
                    playMusic(path + fileArray[songNumber]);
                    showVoiceText.setText("Now playing: " + fileArray[songNumber].replace(".mp3", ""));
                }
            });
            Toast.makeText(getApplicationContext(), song, Toast.LENGTH_SHORT).show();
            mPlayer.setDataSource(song);
            mPlayer.prepare();
            mPlayer.start();
            isPlaying = true;
            } catch (IllegalArgumentException e) {
                e.printStackTrace();
            } catch (Exception e) {
                System.out.println("Exception of type : " + e.toString());
                e.printStackTrace();
            }
        //showVoiceText.setText("Now playing: " + fileArray[songNumber].replace(".mp3", ""));
        }else{
            stopMusic();
        }
    }

We need a void that will stop the music once we ask for it. On the void speechStandBy we were pausing the music so that we could as for the next command we need a void to restart the music after the application gets the void command!

void restartMusic(){
    if(isPlaying && isPaused){
        mPlayer.start();
        isPaused = false;
    }
}
private void stopMusic(){
        if(isPlaying) {
            songNumber = 0;
            isPaused = false;
            isPlaying = false;
            mPlayer.pause();
    }
}

This is a pretty simple code that will prevent the application from crashing when we are closing it

   public void onDestroy(){
        if(t1 != null){
            t1.stop();
            t1.shutdown();
        }
        super.onDestroy();
    }

import android.content.ActivityNotFoundException;
import android.content.Intent;
import android.media.MediaPlayer;
import android.os.Bundle;
import android.speech.RecognizerIntent;
import android.speech.tts.TextToSpeech;
import android.support.v7.app.AppCompatActivity;
import android.view.KeyEvent;
import android.view.View;
import android.widget.Button;
import android.widget.TextView;
import android.widget.Toast;
import java.io.File;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Calendar;
import java.util.Locale;
import java.util.regex.Pattern;
public class HomeActivity extends AppCompatActivity {
    private Button openMic;
    private TextView showVoiceText;
    private final int REQ_CODE_SPEECH_OUTPUT = 143;
    String path = "/storage/sdcard1/Music/";
    String[] fileArray;
    public int songNumber = 0;
    MediaPlayer mPlayer = new MediaPlayer();
    public boolean isPlaying = false;
    public boolean isPaused = false;
    int result;
    TextToSpeech t1;
    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_home);
        showVoiceText = (TextView) findViewById(R.id.showVoiceOutput);
        musicScan();
        openMic = (Button) findViewById(R.id.button);
        openMic.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View view) {
                btnToOpenMic();
            }
        });
        t1 = new TextToSpeech(getApplicationContext(), new android.speech.tts.TextToSpeech.OnInitListener(){
            @Override
            public void onInit(int status){
                if(status == TextToSpeech.SUCCESS){
                    result = t1.setLanguage(Locale.getDefault());
                }else{
                    Toast.makeText(getApplicationContext(), "Feature not supported", Toast.LENGTH_SHORT).show();
                }
            }
        });
    }
    private void btnToOpenMic() {
        speechStandby();
    }
    void speechStandby (){
        if(isPlaying){
            mPlayer.pause();
            isPaused = true;
        }
        Intent intent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);
        intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL, RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);
        intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE, Locale.getDefault());
        intent.putExtra(RecognizerIntent.EXTRA_PROMPT, "Speak Now...");
        try {
            startActivityForResult(intent, REQ_CODE_SPEECH_OUTPUT);
        } catch (ActivityNotFoundException tim) {
        }
    }
    void restartMusic(){
        if(isPlaying && isPaused){
            mPlayer.start();
            isPaused = false;
        }
    }
    @Override
    protected void onActivityResult(int requestCode, int resultCode, Intent data){
        super.onActivityResult(requestCode, resultCode, data);
        restartMusic();
        switch (requestCode){
            case REQ_CODE_SPEECH_OUTPUT:{
                if (resultCode == RESULT_OK && null != data){
                    ArrayList voiceInText = data.getStringArrayListExtra(RecognizerIntent.EXTRA_RESULTS);
                    showVoiceText.setText(voiceInText.get(0));
                    String command = showVoiceText.getText().toString();
                    if(command.equals("exit")) {
                        finish();
                        System.exit(0);
                    }else if(command.equals("music")){
                        if(!isPlaying) {
                            mPlayer.reset();
                            playMusic(path + fileArray[songNumber]);
                            showVoiceText.setText("Now playing: " + fileArray[songNumber].replace(".mp3", ""));
                        }
                    }else if(command.equals("stop music")){
                        if(isPlaying) {
                            stopMusic();
                        }
                    }else if(command.equals("next track") || (command.equals("next song"))){
                        if(isPlaying) {
                            songNumber = songNumber + 1;
                            if(songNumber > fileArray.length - 1){
                                songNumber = 0;
                            }
                            mPlayer.reset();
                            isPlaying = false;
                            playMusic(path + fileArray[songNumber]);
                            showVoiceText.setText("Now playing: " + fileArray[songNumber].replace(".mp3", ""));
                        }
                    }else if(command.equals("previous track") || (command.equals("previous song"))){
                        if(isPlaying) {
                            songNumber = songNumber - 1;
                            if(songNumber < 0){
                                songNumber = fileArray.length - 1;
                            }
                            mPlayer.reset();
                            isPlaying = false;
                            playMusic(path + fileArray[songNumber]);
                            showVoiceText.setText("Now playing: " + fileArray[songNumber].replace(".mp3", ""));
                        }
                    }else if(command.equals("music list")){
                        showVoiceText.setText(Arrays.toString(fileArray) + " ");
                    }else if(command.equals("what time is it")){
                        time();
                    }else if(command.equals("what day is today")){
                        date();
                    }else if(command.toLowerCase().contains("play")){
                        String inputSong = (command.substring(command.lastIndexOf("play") + 5));
                        String inputSongLowerCase = inputSong.toLowerCase();
                        searchSong(inputSongLowerCase);
                    }
                }
            }break;
        }
    }
    private void playMusic(String song){
        if(!isPlaying) {
            try {
               mPlayer.setOnCompletionListener(new MediaPlayer.OnCompletionListener() {
                   @Override
                    public void onCompletion(MediaPlayer mPlayer) {
                        isPlaying = false;
                        mPlayer.reset();
                        if(songNumber > fileArray.length - 1){
                            songNumber = 0;
                        }else {
                            songNumber = songNumber + 1;
                        }
                        playMusic(path + fileArray[songNumber]);
                        showVoiceText.setText("Now playing: " + fileArray[songNumber].replace(".mp3", ""));
                    }
                });
                Toast.makeText(getApplicationContext(), song, Toast.LENGTH_SHORT).show();
                mPlayer.setDataSource(song);
                mPlayer.prepare();
                mPlayer.start();
                isPlaying = true;
                } catch (IllegalArgumentException e) {
                    e.printStackTrace();
                } catch (Exception e) {
                    System.out.println("Exception of type : " + e.toString());
                    e.printStackTrace();
                }
            //showVoiceText.setText("Now playing: " + fileArray[songNumber].replace(".mp3", ""));
            }else{
                stopMusic();
            }
        }
    private void stopMusic(){
            if(isPlaying) {
                songNumber = 0;
                isPaused = false;
                isPlaying = false;
                mPlayer.pause();
        }
    }
    private void musicScan(){
        File f = new File(path);
        File file[] = f.listFiles();
        fileArray = new String[file.length];
        for (int i = 0; i < file.length; ++i){
            fileArray[i] = file[i].getName();
        }
    }
    public void toSpeak(String toSpeak){
        Toast.makeText(getApplicationContext(), toSpeak, Toast.LENGTH_SHORT).show();
        int speak = t1.speak(toSpeak, TextToSpeech.QUEUE_FLUSH, null);
    }
    public void time(){
        Calendar calendar = Calendar.getInstance();
        SimpleDateFormat mdformat = new SimpleDateFormat("HH:mm");
        String strTime = "The time is : " + mdformat.format(calendar.getTime());
        toSpeak(strTime);
    }
    public void date(){
        Calendar calendar = Calendar.getInstance();
        SimpleDateFormat mdformat = new SimpleDateFormat("dd-MMM-yyyy");
        String strDate = "Today is: " + mdformat.format(calendar.getTime());
        toSpeak(strDate);
    }
    public void onDestroy(){
        if(t1 != null){
            t1.stop();
            t1.shutdown();
        }
        super.onDestroy();
    }
    public void searchSong(String songToSearch){
        try {
            Toast.makeText(getApplicationContext(), songToSearch, Toast.LENGTH_SHORT).show();
            for (int i = 0; i < fileArray.length; i++) {
                String value = fileArray[i].toLowerCase().replace(".mp3", "");
                if (Pattern.compile(Pattern.quote(songToSearch), Pattern.CASE_INSENSITIVE).matcher(value).find()) {
                    songNumber = i;
                    isPlaying = false;
                    mPlayer.reset();
                    playMusic(path + fileArray[songNumber]);
                    showVoiceText.setText("Now playing: " + fileArray[songNumber].replace(".mp3", ""));
                    break;
                }
            }
        } catch (IllegalArgumentException e) {
            e.printStackTrace();
        } catch (Exception e) {
            System.out.println("Exception of type : " + e.toString());
            e.printStackTrace();
        }
    }

It is really important to leave a feedback in the comment section to help me improve!