Butterfly Joystick Reading PortE

As a newbie I am learning the Butterfly. I am using Bascom, as I am not a C or ASM jprogrammer.
I am trying to read the Butterfly Joystick status, simplistically, just reading the Ports/bits, without any debounce, without any interrupts.
I CAN easily read the PortB values, I can NOT read the PortE values. I believe I have set the ports for input, turned ON the internal Pull up resistors, and am reading using PinX, (not a port read).
I know the hardware works, as the original Butterfly code which uses the joystick to navigate the menu worked.
I have seen several posts, Bascom, AVRFreaks, etc., that mention trouble reading the PortE joystick values. I have not seen any answers / solutions.
Do I need to set/program any special registers to make sure the PortE.2 and PortE.3 pins are set for simple digital I/O?
I know that the original Butterfly program used interrupts to detect joystick activity. A simple polled bit reading should be easily doable!
The program is long due t the butterfly LCD driver, the key part is the port/pin setup and the polling.

I would GREATLY appreciate any guidance offered!!

JC

CODE

'Purpose: Test reading the Butterfly 5-Position JoyStick
'Set bits for input, read them, be able to use JoyStick as a user interface.
'Keep it simple:
'Polled reading of the joystick input port pins
'No interrupts, No debounce

'LCD Demo counts and displays counter on LCD
'With each time through the loop scan the Joystick pins and write the
'Port & Bit number on the LCD for any activated joystick value.

'Hardware is as defined below.
'Set PORT's directions
'Read Pin's input
'Enable the internal pull-up resistors.
'Joystick switches are normally high, activated = low.

'BF is on a breadboard, with a clean 3.3 V power supply.

'JoyStick Test:
'Make the known ports and bits inputs, and test for activation.
'Determine which port/bit is which direction, as it isn't
'spelled out in the manual.
'JS inputs are Normally High, Activated goes Low
'Up B6
'Down B7
'In B4
'Rht E
'Lft E

'BUG:
'Reading the Butterfly Joystick is proving to be problematic.
'I can read 3 of the 5 positions, by polling the pit status,
'as expected. These 3 are all on port B. The two (L & R)
'events, on Port E, I can not read the joystick correctly. It acts as if the
'internal pull ups are not active.
'I Think I Am Setting Them Correctly.
'If I purposefully turn off the int pull ups on port B, then it acts the
'same way.
'Of interest, the original Butterfly program works, ie can read the joystick, but
'it does it using interrupts.

'Hardware Definition:
'Port B:
'Bit 0 = LED High = ON (My addition)
'Bit 1 = PB Switch, NO, High, Push = Low (My addition)
'Bit 2 =
'Bit 3 =
'Bit 4 = JoyStick, Push In
'Bit 5 = Speaker, Timer1, PWM, Output A, PB5
'Bit 6 = JoyStick, Push Up
'Bit 7 = JoyStick, Push Down

'Port E:
'Bit 2 = JoyStick, Push (R/L ?)
'Bit 3 = JoyStick, Push (R/L ?)

'Port F:
'Bit 0 = NTC on ADC0 Neg Temp Coef Thermister
'Bit 1 = V ADC Input, ADC1, 0-5 V, PF1
'Bit 2 = LDR Input, Light Sensor Input (abscent)
'Bit 3 =
'Bit 4 =
'Bit 5 =
'Bit 6 =
'Bit 7 =

'-----------------------------------------------------------------------------------------

$regfile = "m169def.dat" ' specify the used micro
$crystal = 8000000 ' used crystal frequency
$baud = 19200 ' use baud rate
$hwstack = 32 ' default use 32 for the hardware stack
$swstack = 10 ' default use 10 for the SW stack
$framesize = 40 ' default use 40 for the frame space

$lib "lcd_butterfly.lbx"

'BF LCD Setup:
Dim _butterfly_digit As Byte
Dim S As String * 6

Dim Bb As Bit 'Testing reading joystick values...

'Write a 1 to the Port & Pin, on INPUTs, to Enable Internal Pull-Up resistors:
Set Portb.1 'Set it before config as input, turns on Int Pull Up
Set Portb.4 'Joystick Pull-UP Enabled
Set Portb.6 'Joystick Pull-UP Enabled
Set Portb.7 'Joystick Pull-UP Enabled
Set Porte.2 'Joystick Pull-UP Enabled
Set Porte.3 'Joystick Pull-UP Enabled

'Now Configure the Port's Pins for Input or Output mode.
'0 = Input, 1 = Output
'For Port E am currently interested in pins 2 & 3, (Joystick)
Config Portb = &B00100001 'Now Config PORT Input/Output
Config Porte = &B00000000 'Guessing for Port E other than bits 2, 3

'Alternative Port E Pin Configuration, this also did not work, (Sets just the pins of interest):
'Config Porte.2 = Input 'JoyStick
'Config Porte.3 = Input 'Joystick

'Tried writing to the INPUT Pins after defining the ports input and output mode
'to be sure internal pull up resistors are on.
'This made no difference, still does not work.
Set Porte.2 'Repeat, after port setup , no difference
Set Porte.3 'Or PortE.3 = 1 either is OK

'Butterfly LCD Test:
S = "AVR" 'Display String on LCD
Lcd "HELLO" 'Display Start Up Up Mesage
Waitms 250

Reset Portb.0 'LED Off

Dim J As Byte
Myloop:
For J = 0 To 100
Lcd " " 'Cls LCD by overwrite with spaces
Lcd J 'Display count
Waitms 100
Gosub Joystick 'Test JoyStick
Next
Goto Myloop 'Loop Forever for testing

Joystick:
'Test the Butterfly JoyStick
'Pins should be normally high, low when joystick is activated
'Tried enabling the internal pull up resistors here, also, without effect,
'just before the pins are actually read.
'LCD Displays the Port/Pin if the joystick is activated, then moves on
Set Porte.2
Set Porte.3

If Pinb.4 = 0 Then 'Joystick PortB.4 works, JS is Pushed Inwards
Cls
Lcd "B4"
Waitms 250
End If

If Pinb.6 = 0 Then 'Joystick PortB.6 works, JS is Pushed Upwards
Cls
Lcd "B6"
Waitms 250
End If

If Pinb.7 = 0 Then 'Joystick PortB.7 works, JS is Pushed Downwards
Cls
Lcd "B7"
Waitms 250
End If

If Pine.2 = 0 Then 'Joystick PortE.2 Fails Always reads input as activated, (Low)
Cls 'Always reads input as activated, (Low)
Lcd "E2"
Waitms 250
End If

If Pine.3 = 0 Then 'Joystick PortE.2 Fails
Cls 'Always reads input as activated, (Low)
Lcd "E3"
Waitms 250
End If

'Alternative, assign pin value to a variable, did not make a difference
'Bb = Pine.2 'Read State of INput Pin, made no difference
'If Bb = 0 Then
' Cls
' Lcd "E22"
' Waitms 250
'End If

Return

Lcd_butterfly_data:
Data 0% ' space
Data 0% ' !
Data 0% '""
Data 0% ' #
Data 0% '$
Data 0% ' %
Data 0% ' &
Data 0% ' '
Data 0% ' (
Data 0% ' )
Data 0% ' *
Data 0% ' +
Data 0% ' ,
Data &H0A00% ' -
Data &H8000% ' .
Data 0% '/
Data &H5559% ' 0
Data &H0118% '1
Data &H1E11% ' 2
Data &H1B11% ' 3
Data &H0B50% ' 4
Data &H1B41% ' 5
Data &H1F41% '6
Data &H0111% ' 7
Data &H1F51% ' 8
Data &H1B51% '9
Data 0% ':
Data 0% ';
Data 0% '<
Data &H0004% '=
Data 0% '>
Data 0% '?
Data 0% '@
Data &H0F51% ' A
Data &H3991% , &H1441% , &H3191% , &H1E41% , &H0E41% , &H1D41% , &H0F50% , &H2080% , &H1510% , &H8648% , &H1440% , &H0578%
Data &H8570% , &H1551% , &H0E51% , &H9551% , &H8E51% , &H9021% , &H2081% , &H1550% , &H4448% , &HC550% , &HC028% , &H2028% , &H5009%