"Creating beloved products with beloved technology"

• Founded: 2014 (Taipei)
• Expertise: Software development centered on Ruby/Rails
• Track Record: Specializing in startup systems, also handling government collaborations

1. Contract Development Services

• Taiwan's largest Ruby development company (Founded 2014)
• Infrastructure operations for both cloud and on-premise
• International expansion including Japan, US, and Singapore
• Long-term partnerships from startups to listed companies
• https://5xruby.com/en

2. SOSI Product

• Secure remote access management system
• Bastion server functionality
• Browser-based VDI solution
• https://www.sosi.com.tw

Today's Content

1. The Story of No Bidders

   • Why did nobody dare to bid?

2. What is POJ (Taiwanese Romanization)?

   • Romanization system for Taiwanese

3. Word Segmentation Alignment Implementation

   • Implementation using GSUB

4. Encounter with Parser

   • Re-implementation with Parslet

5. Project Results

   • Ruby's strengths and achievements

6. Conclusion

   • Summary

Slide Materials

https://rwc2025.ryudo.tw

Reasons for Losses (Non-Technical)

• Requirements based on Microsoft products
• "Compatibility requirements" with existing systems
• Opaque evaluation criteria
• Not about price competition, but technology stack constraints

9th Time: Surprising Turn of Events

• Competitors: Zero
• "Why is nobody bidding?"
• Even the project manager was confused: "Are you sure about this?"
• What happened?

Romanization for Taiwanese Language

• Official Name: Taiwan Minnanyu Luomazi Pinyin Fang'an
• Abbreviation: Tâi-lô
• Established: October 2006, promulgated by Taiwan's Ministry of Education
• Status: Official writing system for Taiwanese

Not Mandarin Chinese

• Taiwanese: Minnan language family
• Features:
  • 9 tones
  • Unique consonant and vowel systems
  • Nasalization notation
• History: Developed based on POJ (Pe̍h-ōe-jī) incorporating IPA (International Phonetic Alphabet) elements

Japanese System

Kanji → Hiragana

Correspondence:

• One group of words → One group of kana

Examples:

生活    → せいかつ
新幹線  → しんかんせん
東京駅  → とうきょうえき

Taiwanese System

Kanji → POJ

Correspondence:

• One group of words → One group of POJ

Examples:

紲落      → suà-lo̍h
新竹市    → Sin-tik-tshī
明仔載    → bîn-á-tsài

washed_kanji - Kanji Side

def washed_kanji
  KANJI_GSUB_PATTERNS.reduce(kanji) do |ks, (mt, kp)|
    ks.gsub(mt, kp)
  end
end

Processing:

• Uses dozens of regex patterns for string processing
• Insert spaces around punctuation
• Separate periods, commas, parentheses, etc.

Example:

Input:  做工課的Lín--sàng。
Output: 做工課的Lín --sàng。

washed_roman - POJ Side

def washed_roman
  ROMAN_GSUB_PATTERNS.reduce(roman) do |rs, (mt, rp)|
    rs.gsub(mt, rp)
  end
end

Processing:

• Normalize punctuation with 65+ patterns
• Hyphens are preserved - syllable separators
• Double hyphens (--) also preserved - inter-word pause

Example:

Input:  tsò-khang-khuè ê Lín--sàng.
Output: tsò-khang-khuè ê Lín--sàng .

Implementation Code

def splitted_kanji
  combine_one_word(
    washed_kanji.scan(RXP_SPK).map do |spka|
      spka.split(/\s/)
    end.flatten.join(' ')
  ).split
end

# RXP_SPK - Identifies CJK and non-CJK characters
RXP_SPK = /[\p{Han}\p{Katakana}\p{Hiragana}
  \p{Hangul}\u3000-\u303F\uFF00-\uFFEF]|
  [^\p{Han}\p{Katakana}\p{Hiragana}
  \p{Hangul}\u3000-\u303F\uFF00-\uFFEF]+/x

# combine_one_word - Special combination handling
def combine_one_word(text)
  ONE_KANJI_WORDS.reduce(text) do |ks, (mt, kp)|
    ks.gsub(mt, kp)
  end
end

Processing Explanation

1. Character scan with RXP_SPK

   • CJK characters (Kanji, Hiragana, etc.)
   • Non-CJK characters (POJ, numbers, etc.)
   • Character by character or grouped non-CJK sequences

2. Special handling with combine_one_word

   • Apply ONE_KANJI_WORDS patterns
   • Combine specific symbol sequences

3. Split by spaces

4. Edge Case handling:

   • Lín--sàng is recognized as a single token

Example

Input: 做工課的Lín--sàng 。
Output:
["做", "工", "課", "的", "Lín--sàng", "。"]

Implementation Code

def splitted_roman
  washed_roman
    .split(/\s/)
    .compact_blank
end

Simple! Just 3 lines

Processing Explanation

1. Simple: Split by spaces

   • Phase 1 pre-processing makes Phase 2 simple
   • Can split by spaces alone

2. Important design:

   • Hyphens are syllable boundary markers
   • Don't split by hyphens
   • Preserve double hyphens (--)

3. compact_blank removes empty elements

Example

Input: tsò-khang-khuè ê Lín--sàng .
Output:
["tsò-khang-khuè", "ê", "Lín--sàng", "."]

Syllable counts:

• tsò-khang-khuè = 3 syllables
• Lín--sàng = 2 syllables (-- not counted in syllables)

def roman_kanji_array
  spk = splitted_kanji.dup
  splitted_roman.map do |rword|
    if rword == '--' || (SP_MIRRORS.key?(rword) &&
        #... Edge Case handling
        [rword, spss]
      end
    end
  end
end

def set_arrays
  rka = roman_kanji_array.transpose
  assign_attributes(
    roman_array: rka[0],
    kanji_array: rka[1]
  )
  self.arrays_balanced = [
    roman_array.size.positive?,
    roman_array.size == kanji_array.size,
    kanji_array.join.size ==
      washed_kanji.delete(' ').size
  ].all?
end

Processing Explanation

1. Syllable-based matching

   • Hyphen = syllable separator
   • tsò-khang-khuè (3 syllables) → 3 Kanji characters

2. Edge Case handling:

   • When Roman appears in Kanji side, keep as-is
   • Double hyphen (--) not counted in syllables

3. Balance validation (3 conditions)

   • Array is not empty
   • Roman and Kanji element counts match
   • Total Kanji character count matches original text

Why Parslet?

• PEG Parser: Parsing Expression Grammar
• Ruby DSL: Define Parser using Ruby syntax
• Clear Structure: Naturally implements 3-phase design

# Parslet basics
class MyParser < Parslet::Parser
  # Phase 1 & 2: Rule definition
  rule(:word) { match['a-z'].repeat(1) }
  rule(:sentence) { word >> space }

  root(:sentence)
end

Parslet's Design Philosophy

Parslet makes developers aware of 3 Phases:

Phase 1: Lexical Analysis

• Define token types with rule()
• Character patterns with match[], str()

Phase 2: Syntax Analysis

• Combine rules with >>, |
• Automatically construct AST

Phase 3: Semantic Analysis

• Transform with Transform class
• AST → Final data structure

Punctuation Handling

GSUB Approach

# Part of 65+ patterns
ROMAN_GSUB_PATTERNS = {
  /,/ => ' , ',      # spaces around comma
  /\./ => ' . ',     # spaces around period
  /!/ => ' ! ',      # spaces around exclamation
  /\?/ => ' ? ',     # spaces around question mark
  # ... 60+ more patterns
}

# Application
text = "suà-lo̍h,lâi-khuànn"
ROMAN_GSUB_PATTERNS.each do |pattern, replacement|
  text = text.gsub(pattern, replacement)
end
# => "suà-lo̍h , lâi-khuànn"

Feature: Surround symbols with spaces → split later

Parslet Approach

# Directly recognize punctuation as tokens
rule(:punctuation) do
  str('...') | str('⋯⋯') | str('……') |  # multi-char first
  match[',.:;()!?？！/~、─…⋯'] |         # single chars
  match["\"'\u201C\u201D\u2018\u2019"] |  # quotes
  match['\u3000-\u303F']                  # CJK symbols
end

# Token rules
rule(:token) do
  hyphenated_word.as(:word) |
  punctuation.as(:punct)
end

Input: "suà-lo̍h,lâi-khuànn"

Output (AST):

[
  { word: "suà-lo̍h" },
  { punct: "," },
  { word: "lâi-khuànn" }
]

Feature: Structured as tokens → no split needed

Preserving Hyphens (Page 17)

GSUB Approach

# Step 1: Normalize punctuation
text = "suà-lo̍h lâi-khuànn"
# Preserve hyphens (important!)

# Step 2: Split by spaces
tokens = text.split(/\s/)
# => ["suà-lo̍h", "lâi-khuànn"]

# Step 3: Count syllables
syllables = "suà-lo̍h".split('-').size
# => 2

# Step 4: Take kanji characters by syllable count
kanji_chars = ["紲", "落", "來", "看"]
combined = kanji_chars.shift(syllables).join
# => "紲落"

Principle: Hyphen = syllable separator

Parslet Approach

# Recognize hyphenated word as single token
rule(:hyphenated_word) do
  syllable >>
  (single_hyphen >> syllable).repeat
end

# "suà-lo̍h" → { word: "suà-lo̍h" }

Syllable count calculation:

# Phase 3: Transform
rule(word: simple(:w)) do
  syllables = w.to_s.split('-').size
  # => 2
end

Kanji matching:

# Syllable count = Kanji character count
"suà-lo̍h".split('-').size  # => 2
"紲落".chars.size            # => 2
#  Match!

Principle: Parser preserves syllable structure → automatic matching

Ruby Parser (Prism)

# Input
"def foo(x); x + 1; end"

Phase 1: Lexical

[DEF][IDENTIFIER][LPAREN][IDENTIFIER]
[RPAREN][SEMICOLON][IDENTIFIER][PLUS]
[INTEGER][SEMICOLON][END]

Phase 2: Syntax

DefNode(
  name: :foo,
  parameters: ParametersNode(...),
  body: StatementsNode(...)
)

Phase 3: Semantic

• Type checking
• Scope analysis
• Code generation

Taiwanese Parser (RomanParserPure)

# Input
"suà-lo̍h lâi-khuànn"

Phase 1: Lexical

[suà-lo̍h][lâi-khuànn]

Phase 2: Syntax

{
  sentence: [
    { word: "suà-lo̍h" },
    { word: "lâi-khuànn" }
  ]
}

Phase 3: Semantic

• AST transformation
• Array generation

["suà-lo̍h", "lâi-khuànn"]

Note: Experimental implementation (educational purpose)

POJ Parser (Complex)

# RomanParserPure - Implemented with Parslet
roman_array = [
  "suà-lo̍h",      # 2 syllables
  "lâi-khuànn",    # 2 syllables
  "Sin-tik-tshī"   # 3 syllables
]

# Syllable count calculation
"suà-lo̍h".split('-').size  # => 2
"Sin-tik-tshī".split('-').size  # => 3

Complex processing:

• Hyphen semantic analysis (syllable vs inter-word)
• Tone mark recognition (Unicode combining characters)
• Double hyphen (--) special handling
• Grammar rule definition and AST construction

Kanji Processing (Simple)

# Just follow POJ syllable counts
kanji = "紲落來看新竹市"

# 1. "suà-lo̍h" = 2 syllables
#    → 2 Kanji chars: "紲落"
# 2. "lâi-khuànn" = 2 syllables
#    → 2 Kanji chars: "來看"
# 3. "Sin-tik-tshī" = 3 syllables
#    → 3 Kanji chars: "新竹市"

kanji_array = ["紲落", "來看", "新竹市"]

Simple processing:

• Syllable count = character count correspondence
• Pattern matching (for Edge Cases)
• No independent syntax parsing needed

https://github.com/ryudoawaru/rwc2025-slide

What's included:

• Complete RomanParserPure implementation
• WASHING_PATTERNS (65+ rules)
• 3000 real corpus data records

$ ruby test_parser.rb

================================================================================
Testing RomanParserPure with 3000 records
================================================================================
[██████████████████████████████████████████████████] 100.0% (3000/3000)

================================================================================
Final Results
================================================================================
Total records:    3000
Parse success:    3000 (100.0%)
Parse errors:     0 (0.0%)
================================================================================

 PERFECT! 100% success rate achieved!

Key Points:

• 100% Parse success rate - All 3000 records parsed accurately
• Zero errors, fully functional - Theory meets practice