Java Accessible Object Dialog Problem

Creating macros. Working with QM.

Java Accessible Object Dialog Problem

Postby tanhy on Mon May 01, 2017 4:36 pm

Greetings. I am using Quick Macro v2.4.5 (for the first time :) ) in the hopes to automate a Java JNLP file to print a page inside using solely non-keyboard/mouse actions so that I can run it in the background and on a lock computer. But I have encountered problems with QM recognising subsequent dialog boxes if these are opened by QM accessible object actions.

I have Java (32-bit) installed, have enabled Java Access Bridge and can use accessible objects to find whatever buttons I need to find during the initial programming phase. The only action I have used is "DoDefaultAction" as these buttons are push buttons.

One example:
So the problem is that if I use QM accessible object actions (per below script) to open a dialog box (say, Help -> About), the "About" dialog box will open/show. But there is something strange with the opened "About" dialog box. The "Close" button in the opened "About" dialog box cannot be clicked via the last section of the codes below and return an error "Error (RT) in Macro 1: object not found. ?" at the 'Acc a2' line. For other dialogs, it sometimes works but with a great amount of delay.

int w=win("Login - Java Terminal" "SunAwtFrame")
Acc a.Find(w "menu" "Help" "" 0x1001)
a.DoDefaultAction
Acc a1.Find(w "menu item" "About" "" 0x1001)
a1.DoDefaultAction

int w1=win("About Java Terminal" "SunAwtDialog")
Acc a2.Find(w1 "push button" "Close" "" 0x1001)
a2.DoDefaultAction



So the thread ended. Now on the already open "About" dialog (opened using the previous/above QM accessible object action - DoDefaultAction), if I use QM to find accessible objects, it seems like only the windows can be found and not individual objects within the window. The QM program then hung badly. For the times it managed to recover, instead of being able to locate the dialog object, it ends up showing some WINDOW (SunAwtDialog). This is unlike the initial programming phase where I manually opened the dialog in the JNLP program prior to using QM to (successfully) find these accessible objects.
Also, on the already open "About" dialog (opened using the previous/above QM accessible object action - DoDefaultAction), I am able to use my mouse to interface directly with the JNLP program to manually click the "Close" button to close the "About" dialog box.


Now if I use keyboard actions in QM (per below script) to open the "About" dialog box, the same QM accessible object actions in 'Acc a2' line was able to find and execute the "Close" button within the "About" dialog box and thereby closed this dialog without delay.

int w=act(win("Login - Java Terminal" "SunAwtFrame"))
'Ah ;; Alt+H
'Aa ;; Alt+A

int w1=win("About Java Terminal" "SunAwtDialog")
Acc a2.Find(w1 "push button" "Close" "" 0x1001)
a2.DoDefaultAction


Also, this problem is only encountered on "SunAwtDialog" class (and if this dialog was opened using QM accessible object actions). It does not occur on "SunAwtFrame" class regardless of the method used - mouse, keyboard or accessible object actions.

Any advice on a workaround to this issue? I would strongly prefer to get the "accessible object" method to work rather than requiring to unlock the computer and using keyboard actions. Otherwise are there any other methods to click dialog box buttons using non-keyboard/mouse actions?
Thanks.
tanhy
 
Posts: 3
Joined: Mon May 01, 2017 2:54 pm

Re: Java Accessible Object Dialog Problem

Postby Gintaras on Tue May 02, 2017 6:07 am

Thank you. I tested with the Java Control Panel app. When clicked a button to open a dialog, DoDefaultAction opens the dialog, but then error, and Java accessible objects stop working. Cannot use DoDefaultAction until Java will fix it. Workaround:

Macro Macro2851     ?
int w=win("Java Control Panel" "SunAwtFrame")
Acc a.Find(w "push button" "Network Settings..." "" 0x1001)

;a.DoDefaultAction ;;bad
;a.JavaAction("click") ;;bad

;workaround
a.Select(1)
;SendKeysToWindow w key(V) ;;if you have it. Else:
PostMessage w WM_KEYDOWN VK_SPACE 0x390001; 0.02
PostMessage w WM_KEYUP VK_SPACE 0xC0390001; 0.02

int w1=wait(1 WV win("Network Settings" "SunAwtDialog"))
Acc a1.Find(w1 "radio button" "Use proxy server" "" 0x1001)
a1.DoDefaultAction
0.5
int w2=win("Network Settings" "SunAwtDialog")
Acc a2.Find(w2 "push button" "Cancel" "" 0x1001)
a2.DoDefaultAction



rather than requiring to unlock the computer

The script probably will not work when computer is locked. Any similar script would not work. Then even cannot find windows.
This script works when the Java window is not the active window. But its taskbar button becomes red...
Gintaras
Site Admin
 
Posts: 10385
Joined: Thu Dec 19, 2002 7:36 pm
Location: Lithuania

Re: Java Accessible Object Dialog Problem

Postby tanhy on Thu May 04, 2017 3:45 pm

Thank you very much. Had to opportunity to try it tonight.
Excellent, it works! :D I am able to use PostMessage on a lock computer.

Do you know what is the issue with Java that caused DoDefaultAction and JavaAction("click") to not work?

With regards to the PostMessage, how did you get the "0x390001" and "0xC0390001" parameters? I believe that is the lParam parameters but I do not understand how to obtain its value.

And in my Java JNLP file, there are a few button that does not support the Select key (one of which is the "Print" button). In particular, the Tab key does not cycle through it. Any advice how then can I use the workaround for such buttons?

In any case, for the "Print" button specifically, I am able to use Ctrl+P to trigger it. I tried the following script but it seems to not be inputting the Ctrl key and only managed to sent the P key alone. There is no difference if I change VK_CONTROL to VK_LCONTROL or VK_RCONTROL and 0x50 to 'P'. Is there something wrong with the below script?

int w=wait(1 WC win("Active - Java Terminal" "SunAwtFrame"))
Acc a.Find(w "push button" "Print" "" 0x1001 1)
PostMessage w WM_ACTIVATE WA_ACTIVE 0
PostMessage w WM_KEYDOWN VK_CONTROL 0x390001; 0.02
PostMessage w WM_KEYDOWN 0x50 0x390001; 0.02
PostMessage w WM_KEYUP 0x50 0xC0390001; 0.02
PostMessage w WM_KEYUP VK_CONTROL 0xC0390001; 0.02
tanhy
 
Posts: 3
Joined: Mon May 01, 2017 2:54 pm

Re: Java Accessible Object Dialog Problem

Postby Gintaras on Thu May 04, 2017 5:01 pm

Do you know what is the issue with Java that caused DoDefaultAction and JavaAction("click") to not work?

Don't know, but this helped me to understand partially: https://community.oracle.com/thread/2595410
The Java DoDefaultAction function does not return until the dialog is closed. If somebody/something closes it within several seconds, the macro can continue.

With regards to the PostMessage, how did you get the "0x390001" and "0xC0390001" parameters?

It is documented in MSDN library. Function SendKeysToWindow2 (below) calculates them. But I simply took them from Spy++, it is a program installed with older versions of Visual Studio and maybe can be downloaded separately, or you can find similar programs. It monitors and decodes Windows messages received by a window. Useful in automation.

To send keys to an inactive window better use this function. It translates QM key codes and posts messages. Also it supports modifier keys (Ctrl etc) which cannot be simply posted like other keys. Not very reliable, with some windows or in some conditions may be too fast, then try to make it wait longer after posting messages.

Function SendKeysToWindow2     ?
;/
function hwnd ARRAY(KEYEVENT)'a

;Sends keys to a child window using WM_KEYDOWN/WM_KEYUP messages.
;The window can be inactive, but with some windows then does not work.

;hwnd - child window handle. Or main window handle, if it receives keys.
;a - the <help>key</help> function.

;EXAMPLE
;int hwnd=id(15 "Notepad")
;SendKeysToWindow2 hwnd key("text[]" Cs)


int i lparam m up alt ati
str sk
ifk(A) alt=1

for i 0 a.len
,KEYEVENT k=a[i]
,if(k.flags&0x80) i+1; continue ;;wait
,sel(k.vk)
,,case [16,17,18]
,,int th1=GetCurrentThreadId
,,int th2=GetWindowThreadProcessId(hwnd &i)
,,ati=AttachThreadInput(th1 th2 1)
,,sk.all(256)
,,0.01
,,break

for i 0 a.len
,k=a[i]
,
,if(k.flags&0x80) ;;wait
,,i+1
,,opt waitmsg -1
,,wait (a[i].wt/1000.0)
,,continue
,
,lparam=k.sc<<16|1
,if(k.flags&KEYEVENTF_EXTENDEDKEY) lparam|0x01000000
,if(k.flags&KEYEVENTF_KEYUP) lparam|0xC0000000; up=1; else up=0
,
,if(alt) m=iif(up WM_SYSKEYUP WM_SYSKEYDOWN); lparam|0x20000000
,else m=iif(up WM_KEYUP WM_KEYDOWN)
,
,if(k.vk=VK_RETURN) 0.01 ;;some controls, eg PSPad, would insert new line too early
,
,PostMessage hwnd m k.vk lparam
,0.01
,
,sel(k.vk)
,,case [16,17,18] ;;modifier keys
,,;wait while processes prev keys
,,if(i) SendMessage hwnd 0 0 0
,,0.01
,,
,,GetKeyboardState sk
,,sk[k.vk]=iif(up 0 0x80)
,,SetKeyboardState sk
,
,if(k.vk=VK_MENU) alt=!up

if(ati) ati=AttachThreadInput(th1 th2 0)



Tested with the same Java window and it works.

Macro Macro2854     ?
int w=win("Java Control Panel" "SunAwtFrame")
;act w
Acc a.Find(w "push button" "Network Settings..." "" 0x1001)

;a.DoDefaultAction ;;bad
;a.JavaAction("click") ;;bad

int isActive=w=win
if(!isActive) SendMessage w WM_ACTIVATE WA_ACTIVE w ;;let the window think it is active, else it ignores the keys
SendKeysToWindow2 w key(An)
if(!isActive) SendMessage w WM_ACTIVATE WA_INACTIVE 0

0.5
int w1=win("Network Settings" "SunAwtDialog")
Acc a1.Find(w1 "push button" "Cancel" "" 0x1001)
a1.DoDefaultAction


;Also tried to post mouse click messages, but it does not work with Java windows.

Gintaras
Site Admin
 
Posts: 10385
Joined: Thu Dec 19, 2002 7:36 pm
Location: Lithuania

Re: Java Accessible Object Dialog Problem

Postby tanhy on Sun May 07, 2017 2:42 pm

Once again, thank you very much for the great assistance. It works!

The SendKeysToWindow2 and basic PostMessage functions managed to allow keys to be entered behind a lock computer thereby working around the Java dialog accessible object actions issue. And the 0.5s delay works great with my Java JNLP file. I have now managed to automate the Java JNLP file. Perhaps the only thing that can be improved is to figure a way to run it in the background. But that said, I am very satisfied with the current macro.

Thanks again for the help and cheers. :D
tanhy
 
Posts: 3
Joined: Mon May 01, 2017 2:54 pm

Re: Java Accessible Object Dialog Problem

Postby macrobear on Fri Oct 13, 2017 9:57 am

Dear All,

Can someone please give me an example how to send keys to an Inactive browser like Chrome or Firefox?

Apparently the code on this page is the best to send to an inactive window, but I cannot figure out how to use this function?

Can you please give the full code?

Much appreciated :D
macrobear
 
Posts: 17
Joined: Tue Nov 01, 2016 6:33 pm

Re: Java Accessible Object Dialog Problem

Postby Gintaras on Fri Oct 13, 2017 10:35 am

This function does not work with inactive Firefox and Chrome. I don't know other ways to send keys to inactive windows.

Macro Macro1044     ?
int w=win(" - Mozilla Firefox" "MozillaWindowClass")
;int w=win(" - Google Chrome" "Chrome_WidgetWin_1")
SendKeysToWindow2 w key("quickmacros.com" Y)


does not work
Gintaras
Site Admin
 
Posts: 10385
Joined: Thu Dec 19, 2002 7:36 pm
Location: Lithuania

Re: Java Accessible Object Dialog Problem

Postby macrobear on Fri Oct 13, 2017 10:46 am

Thank you Gintaras, what about sending only shortcut keys like Ctrl+T or Ctrl+C to inactive browser?
macrobear
 
Posts: 17
Joined: Tue Nov 01, 2016 6:33 pm

Re: Java Accessible Object Dialog Problem

Postby Gintaras on Fri Oct 13, 2017 10:58 am

I did not test.
Gintaras
Site Admin
 
Posts: 10385
Joined: Thu Dec 19, 2002 7:36 pm
Location: Lithuania

Re: Java Accessible Object Dialog Problem

Postby macrobear on Fri Oct 13, 2017 11:04 am

Ok I'm still confused using:

int w=win(" - Mozilla Firefox" "MozillaWindowClass")
;int w=win(" - Google Chrome" "Chrome_WidgetWin_1")
SendKeysToWindow2 w key("quickmacros.com" Y)


Quick Macros is asking about SendKeysToWindow2 & I see it is in the upper part of this page, but do I just add SendKeysToWindow2 above the line
function hwnd ARRAY(KEYEVENT)'a
?
macrobear
 
Posts: 17
Joined: Tue Nov 01, 2016 6:33 pm

Re: Java Accessible Object Dialog Problem

Postby Gintaras on Fri Oct 13, 2017 11:08 am

Create function SendKeysToWindow2 and paste that code.
Gintaras
Site Admin
 
Posts: 10385
Joined: Thu Dec 19, 2002 7:36 pm
Location: Lithuania

Re: Java Accessible Object Dialog Problem

Postby macrobear on Fri Oct 13, 2017 11:25 am

function SendKeysToWindow2
function hwnd ARRAY(KEYEVENT)'a


is not working - how do you enclose one function within another as there is no End Function usage in QM?

Please I'm still very new to this code? :roll:
macrobear
 
Posts: 17
Joined: Tue Nov 01, 2016 6:33 pm

Re: Java Accessible Object Dialog Problem

Postby Gintaras on Fri Oct 13, 2017 11:28 am

Click the ? link in the SendKeysToWindow2 post.

Function SendKeysToWindow2 ?
Gintaras
Site Admin
 
Posts: 10385
Joined: Thu Dec 19, 2002 7:36 pm
Location: Lithuania


Return to General

Who is online

Users browsing this forum: No registered users and 1 guest

cron