Progress Senior Project - 3/23/2010

Yeah! I'm finish!

here is our poster for present... Actually, i don't like the way teacher ask us to present via poster. It was very bored. Anyway, it wasn't too bad, I can looking around and see other project.

Yeah! It maybe my last blog for project. T^T kind of sad but i'll continue write more about my work next time. :-P

Progress Senior Project - 3/15/2010

Final video with 3 experiments.

The last experiment is another form of Thai music sheet. To remove the line I use hough transform to remove it but after that, it make the program can't work well on reading but it still acceptable(for me)

Next time, I'll show you the poster for our presentation. See you next time.

Progress Senior Project - 3/02/2010

All my false, sorry.

I got a lot of things to done this period. Anyway, now i just remain only exam and senior project. Last week and this week is very illness. I was sick, sleep late, lazy and bored. I didn't go home for a month. (OMG!!)

It has the information as much as this blog but the report is seem to be more formal.

All my false, I forgot to check the report from my friends. There are a lot of missing and mistake. I can't find the editable version of report, can't even find the final report in my computer. O_o...

Aj. Nuttanart just return the report on Wednesday and we need to hand in within this Tuesday which Monday is a holiday. ^^"" I worked on the addition part which were appendix and theory. Then send it to my partner on Saturday morning (See! I can't went to the bar and can't went to see my friends from Changmai). But my partner just sent the full report to adviser on Monday. (T T) Good luck everyone, I saw a lot of my friendssss still awake at 5 am haha!!! LOL

Progress Senior Project - 2/15/2010

Actually it could be my last day for the eight precepts... But because of Chinese New Year, make me stop doing for 3 days (OMG!!!) I'll find another day to make up these 3 days.

As I already told you about the interface, today I work more about it. Here's the video test run of my program. How was it?

Here is the interface document of today: https://docs.google.com/fileview?id=0B-Ls4RhAgiHeZTFjYmEwMDAtZjI5Mi00MDIyLWIyMDgtNjBhMDFlZTM3MWM3&hl=en

So, next task, my program will have>>

• more efficiency on OCR
• can remove the table line
• select beat of the song
• make XML file

And this is just optional if i have time

• more and more efficiency on OCR
• more efficiency on interface
• select music instrument

Now is 3 am................. My heart beat very fast....... Am I gonna die TwT..... Happy Valentine

Progress Senior Project - 2/13/2010

Happy Chinese New Year!!!!

I just back from my house for celebration. So I started to work very late. (And i kind of mad with my dad and my bro }:( , sorry about that) My dad said, he had read this blog and he don't understand what I wrote. So it make me think like, "Do I wrote bad?". No! I'm not. My word is too easy I know. Anyway, maybe it's just too much technical term.....

So last time I mention about the rest sign (-) wasn't recognized. I think today i can fix it but just for this input because when i use with another input, it was very bad.

Another problem that I found is my MIDI program can't read the notation from the OCR part....

OMG!!!! How can I gonna fix it. You know, I spend all Friday night to fix it seem not work. But this evening, I try it use the encoded character and thank god!!!! it's work. Here is the UTF-8 code for ด - ท.

str('\xb4\n'): #ด

str('\xc3\n'):#ร

str('\xc1\n'):#ม

str('\xbf\n'):#ฟ

str('\xab\n'):#ซ

str('\xc5\n'):#ล

str('xb7\n'):#ท

Here is my program running from part 1 to part 2. (Hey! it's real not "printf" trick na) And next time it will come up with interface and more efficiency on OCR.

PS. I didn't work on circle detection thing yet but I try to study more about it. :-)

Progress Senior Project - 2/11/2010

Today, i found that if database and the image is exactly the same font and size, the result will be 100% correct in template matching.

input

output

This is the result from the song Lao seang Tean (ลาวเสี่ยงเทียน), as you see, it nearly 100% correct but there are no rest sign (- -"). I try to fix it out. I think it should be the error from row segmentation because when i have only one line with no high dot, the result can run normally.

I nearly end (my life @_@) . It's too hard for me.

Today I try to use Hought transform to detect the circle of the image. It work quite weird today, maybe it's a human error. I'll explain more about this thing this evening. Bye G-nite.

PS here is the picture of the notation that make from my project. Actually, the midi file can read in any music notation reader....(How can I forgot this thing!!!) So this way I think making XML file is no need anymore! ^^

Progress Senior Project - 2/9/2010

I feel very sorry to let my worth time with anything that waste my time. There are many thing I have to do during this period, I got SMEs work, job interview, lab, music performing and so on. (But the biggest thing that damage my time is Facebook, so I stop using Facebook for 2 days and continue my work.) After that, I have to hand in the "Final Report" with in 15 Feb. ??? How can I gonna finish in this period of time.

So today, i have to finish another part of my work, here is the explanation. In Thai note, we separate the measure by using | or table like this -v

Normally we using table as the measure so we have to remove table before we go to the OCR part. So the result should be like this -v

After we have the sheet note with out table line, the blob and OCR will work more properly.

My OCR still can't work well hope it will be better with in this week T T.....

Progress Senior Project - 1/14/2010

PA. In the house !!!! (What?)

Today I just make the interface for my project (it nearly to be my project T T)

The process is like I told you in the previous blog.

As you see from the picture, if you click on "import image", it will ask you to open the image file. Then when you click on Making note, the note will show on the process box. Then you can make it play the note, export to midi file or export to XML file as you want. In the rest of this day, I'll make it can export to XML file. As I saw from another midi program, the XML for 1 measure should be like this

From XML file, in one measure there are attribute and note.

note have pitch, duration, type and steam default

pitch have step and octave

for the pitch and the related of step and octave you can read from my older blog.

Finally, I have to hand in the progress report this Monday, do you think this work is enough for the short progress report.

PS. I still waiting for the report from my friend.

Progress Senior Project - 1/9/2010

OMG! How can I left my project for so long, it's about 4 days T T. Actually, I didn't completely left it but I still find the information for making GUI for Python. You know, this week i have a lot of event to attend such as facebook, twitter and so on. (Just kidding!) Actually, I went to do the job interview, making the duet violin score for University song (Yeah, all of the violin student can't arrange or harmonize the song,me either. But I'm the one who understand the notation rule. You know what? I spend whole day but we didn't use it O_o) and preparing for exam, I really apologize to myself in this case.

Anyway, what I have found is the good GUI maker, that is pyQT, wxPython and Tkinter. In this time I didn't have the decision yet but at least I know the way to start. (Yeah ^^) The following picture is the example program for both tool which running on my computer(window xp). I didn't take pyQT because I think the left 2 tools is more popular. Actually, to make the comparison, you have to make it in the same function but I still can't make it so this is the example code from the tutorial.

What do you think? Most of comment said wxPython is better, in this time while i wrote this blog, i'd said so... Maybe this one.

• wxPython
• 
• Tkinter
• 
  
  
  
  

Progress Senior Project - 1/5/2010

I worked on OCR part tonight. Today I made the program can recognize the above and under charactor ( ํ and ฺ ). I think it nearly complete, now it just like 75% correctness.

I think tonight I'll study more about Ocrad [http://www.gnu.org/software/ocrad/ocrad.html]

Actually, I want to work on the interface or makefile part but I don't know how to start. If anyone know the method please tell me. _/|\_

Progress Senior Project - 1/3/2010

Actually I done this part before new year, but i'm in mid-term exam, so i write it a bit late.

This time I worked on the OCR part. It just a template matching, not so good but it almost done (I hope). From the picture, I recognize the character and write it to txt file, and then made the midi sound from the file. The blob coloring part is quite good but some extra character such as -ํ or -ฺ is not good, I'll try it tonight.

Last time I met my advisor, he ask us to make the document, so I let my friend manage the layout but I still don't have it O_o OMG........

Progress Senior Project - 12/9/2009

Today I just work more for my MIDI part. Now my program can play different patch sound. This Example is the sound of flute and xylophone, it can play simultaneously. This song is combining of 2 different song for show that it can play 2 line in 1 track.

I think, before new year we just waiting for OCR part from my friend. T T

Example song: http://www.4shared.com/file/168766436/4477375d/remix.html

GM sound patch :http://www.midi.org/techspecs/gm1sound.php

Progress Senior Project - 12/5/2009

Today I just made my song can play rhythm and melody simultaneously. I use melody as channel 0 and the drum as channel 9

Mary has a little lamp <<<[listen]>>>

I still have some problem about this, because the rest sign (-) need to play the previous note continuously but I can't make it yet.

Laodungdern <<<[listen]>>>

Next, I need the soundbank of Thai flute. My senior friend said he has one, so my next problem is how can I use it properly. If someone know the method, please tell me. I hope I can finish this part within new year... (_/|\_ สาธุ)

Progress Senior Project - 12/1/2009

This week, we seperate the work into 2 part. First the OCR thing, I let my friend take care of it and second is MIDI.

Today, I just write program to generate the sound, that's Marry have a little lamp. You know what, I heard that Thomas Edison made the first recording of the human voice in 1877 with this song. I don't know why I selected this song but when I work with the sound or playing music, I always start with this song. By the way, next I'm going to make the song able to have the rest sign (-) and then make the rhythm. Actually, I think I already done this part but my problem is how to use the Thai sound bank. I hope I can finish this part within this weekend. So let see the flow...

Oh! we forgot to tell you that we decided to work with Python because I already learn about image processing by using Python. For making python can work with midi, I use midipython library (link http://www.mxm.dk/products/public/pythonmidi) and for image processing, I use openCV library (link http://opencv.willowgarage.com/wiki/).

As I tell you on the Progress report last month, after we have the music characters then we make sound from it. Midi represent the sound by the number and sequence like

• midi.note_on(channel=0, note=do)

So to make the sound we have to define the number of note (from ลฺ till ซํ)

• a3=57 (ลฺ)
• b3=59 (ทฺ)
• c3=60 (ด)
• d3=62 (ร)
• e3=64 (ม)
• f3=65 (ฟ)
• g3=67 (ซ)
• a4=69 (ล)
• b4=71 (ท)
• c4=72 (ดํ)
• d4=74 (รํ)
• e4=76 (มํ)
• f4=77 (ฟํ)
• g4=79 (ซํ)

Then we make the event for the song and generate into sound.

• midi.update_time(time)
  midi.note_on(channel, note)
  midi.update_time(time+sequence)
  midi.note_off(channel, note)

Sound easy, isn't it? I think so but the important point of this project is OCR. Now we just work on blob coloring. I hope that we can recognize the character soon.

Now I feel like I have done a little bit farther. ^^ And oh! this is my first MIDI song -> http://www.mediafire.com/?iymjnwzjncz I hope it would be better next time.

Project Proposal

We have added some information about the OCR and MIDI theory. We also have the platform of reading Thai music notation too.


[download]Progress report[front page]: http://docs.google.com/fileview?id=0B-Ls4RhAgiHeNmY0YjdiMzUtMGMxOC00NzcyLWFiNWMtNTYwZDNiNWJhZWNi&hl=en




Progress report: http://docs.google.com/fileview?id=0B-Ls4RhAgiHeMThjZDI3NTItNjJmNy00ZTQ3LThkYTItZTE5ZTJmZmQxMWY5&hl=en


////////////////////////////////////////////////////////////////////////////////////////

Thai music notation Platform


2.1 Thai Music Notation Thai music notes can be represented by using symbols to define the voice manners. The 3 basic symbol sets consist of 1. Pitch of the note. 2. Duration of the note. 3. Other special symbols. 1) Pitch Pitch of Thai music is determined by 7 characters of Thai language plus pitch (.) which is placed below the characters to represent low voice, and the a high-point (.) which is placed over the characters to represent high voice as you can see in the table below.

Table 2.1 The notation representation

Low voice	General Thai music voice	High voice
ดฺ (Low Dole)	ด (Dole)	ดํ (High Dole)
รฺ (Low Re)	ร (Re)	รํ (High Re)
มฺ (Low Me)	ม (Me)	มํ (High Me)
ฟฺ (Low Fa)	ฟ (Fa)	ฟํ (High Fa)
ซฺ (Low Sol)	ซ (Sol)	ซํ (High Sol)
ลฺ (Low La)	ล (La)	ลํ (High La)
ทฺ (Low Si)	ท (Si)	ทํ (High Si)

2) Duration The duration of the Thai note is indicated by measures of the duration. 2.1) Measure Measure performs as beat in international music. It will move forward slow and fast constantly. Every note recording line contains 8 music rooms where the rhythm always will be placed in the end of Thai note in each music rooms.

1

2

3

4

5

6

7

8

2.2) Note arrangement Each music room includes 4 sub-unit beats..

1234

1234

1234

1234

1234

1234

1234

1234

In Thai music notation system, each Thai music note and each symbol –(minus) equal 1 length of rhythm. 

-- equals –ด 

-ด equals รด

--ด equals -รด 

ม-ด equals -มด

-ม-ด equals -มรด 

When the symbol - (minus) is placed on the end of any letter it will extend the sound of the notes 1 sub-unit beat. 

For example: 

ม/---ช/---ล/-ซ-ล/ 

Me---Sol----La---Sol---La 

1234/1234/12/121/ 

Both Thai music note and symbol – (music) can swap their positions follow the melody of the music. Therefore, the symbol arrangement can be in many forms as you can see in an example below.

Case1 : 1 note per room

/---ซ / --- ม / --- ร / ---ด / 

Case2 : 2 notes per room

/ -ล-ล / -ม-ล / -ล-ซ / -ม-ล/ 

Case3 : 3 notes per room

/ -มซม / -ดมร / -มซด / -ลลล /

Case4 : 4 notes per room

/ ซลดร / มรดล / ชลดร / รรรร /

[Senior Project] OCR [1]

OCR

There is several ways to use image processing on OCR program, but we decided to use these 2 algorithms.

1. Blob Coloring

We are using Blob coloring technique for OCR. Once an image has been segmented into as number of regions using thresholding or another method, it is often useful to identify each of the connected components in the image. The process of assigning each distinct region unique identifiers is called blob coloring. Blob coloring program uses a simple recursive search algorithm to find all points connected a starting point. Several program options enable the user to search for white objects on black backgrounds or vice versa, and vary the definition of "connected" to use 4 neighbors or 8 neighbors. (But we use 4 neighbors)

Blob coloring algorithm has two passes. In the first pass, colors are assigned to image pixels by using 4 neighbors, L-shaped mask. Color equivalences are established and stored, when needed. In the second pass, the pixels of each connected region are labeled with a unique color by using the color equivalences obtained in the first pass.

After we have the color equivalences of all character, we draw the crop line of each character for define the character.

2. Matching with database

For the database character, we have 7 character of each tone and define the ration of width and height and group them by the result. (More than 1, 0-1, less than 1) Because it can simplify for compare with the scanned text.

After we have cropped character, for each character we find the ratio of width and height. When we have the ratio of scanned character, then we match the character with the database character in the same group of ratio. (From this algorithm, we can eliminate some character from database) After that, we use sum square error for recognize the most similar character between scanned texts and database.

[Senior Project] Project Proposal

Introduction

Nowadays, Thai musical Technologies are not released much, although many innovations are created, but most of them do not involve about Thai Cultural. Thus for promoting Thai Music to be known widely, we develop software for reading Thai Music Notation to facilitate users.

This project is created to facilitate for users who want to learn about Thai Music. Thai Music Notations is presented in form of Thai Characters. For program processing, program gets input in form of image file from scanning actual notations. After that, it will passed through Optical Character Recognition (OCR) to transform data to characters and display especially inform of sound by generating e and instruction sound to play the note following the songs.

OCR is a program that is developed to transform from images to characters. English version of OCR can recognize efficiency about 99%, but in Thai version still low integrity. This project is applied to be more useful apart from reading only Thai characters.

Objective

1. To apply the knowledge about OCR to implement project

2. To develop software to reading Thai Music Notation

3. To conserve Thai Cultural with using technologies

• PDF File>>http://www.mediafire.com/?nkhwnm05y0o

[Senior Project] Project Detail

Thai Music Notation Reader

Advisor Prof. Kurt Rudhal

Member

Miss Patama Ranchune	Student ID 49270621
Miss Sitanan Nipatvaranan	Student ID 49270650

Project Detail

Thai Music Notation Reader is a program which is implemented for generating Thai Music Notation and transforms to sound. Initially,image processing is utilized in reading Thai Music Notation using OCR which will compare generated notations with the music notations in database.Afterthat, Thai Music sound which matchs with the notation will be played.

To explain Thai Music Notation Reader, we seperate basic information into 3 parts which are easy to understand.

§ Thai Music Notation Behavior

- Thai Music Notations are represented by 7 Thai characters. --> ด ร ม ฟ ซ ล ท (Do Re Me Fa Sol La Ti)
- Quadruple meter (4 accents) are used in separating.
- There are 3 ranks of sound speed (fast, normal and slow)that can define by small cup-shaped symbols called "Ching" accent.
- The octave tone defined by dot(.) behind and below the characters.

§ Program Functions

- Read notation in sequence using OCR.
- Define sound speed.
- Selects the musical sound.
- Selects the language (Thai/English)in user interface.

§ Optional

- Be able to use with USB MIDI controller.
- Be able to play serveral musics simutaneously.